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ABSTRACT 

An  algorithm  for  solving  Dantzig's  generalized  programming  formula¬ 
tion  of  continuous-time  linear-system  optimal  control  problems  is  devel¬ 
oped.  Dantzig's  work  is  extended  to  include  continuous-time  versions  of 
quadratic  loss  criteria  and  minimum  fuel  problems.  New  results  in  param¬ 
etric  linear  and  quadratic  programming  problems,  where  the  parameter 
dependence  is  nonlinear,  are  derived  with  internal  schemes  to  avoid 
cycling  due  to  degeneracy.  Finite  switching  results  in  the  completely 
linear  system,  including  the  minimum  fuel  and  minimal  time  problems, 
are  presented  without  assuming  Pontryagin's  general  position  principal 
or  uniqueness  properties.  The  procedure  initially  finds  a  feasible  and 
admissible  solution  to  the  continuous- time  control  problem  without  using 
discrete  approximations.  The  algorithm  continues  to  converge  monoton- 
ically  to  the  optimal  solution  while  remaining  feasible  and  at  each 
stage,  provides  a  bound  on  the  value  of  the  loss  function  for  termina¬ 
tion  purposes.  This  procedure  is  well  suited  for  systems  with  a  rela¬ 
tively  high  number  of  state  variables  and  control  inputs  for  which  dis¬ 
crete  time  linear  or  quadratic  programming  models  become  too  large. 
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Chapter  I 

INTRODUCTION 

With  the  advent  of  efficient  and  large-scale  mathematical  program¬ 
ming  techniques,  computationally  feasible  methods  are  available  for  op¬ 
timal  control  problems.  The  purpose  of  this  paper  is  to  present  an 
algorithm  for  solving  continuous -time  optimal  control  problems  with 
linear  dynamics  and  various  loss  criteria.  Due  to  the  mathematical 
programming  techniques  used  in  the  algorithm,  it  is  well  suited  for 
large-scale  control  problems,  i.e.,  control  problems  with  large  numbers 
of  state  variables  and  time-varying  control  inputs.  This  work  consists 
of  two  main  results  that  are  combined  to  develop  the  algorithm. 

In  Chapter  II,  we  describe  the  types  of  control  problems  considered, 
including  basic  definitions  and  notations  for  these  problems.  The  basic 

results  in  control  theory  and  certain  necessary  conditions  for  optimal 

v 

control,  as  described  by  Pontryagin  et  al.  [1],  are  also  presented. 

In  Chapter  III,  the  algorithms  and  basic  theorems  for  linear  pro¬ 
gramming  and  the  simplex  method  [2],  quadratic  programming  and  the 
complementary  pivot  theory  [3],  and  the  Dantzig-Wolfe  generalized  pro¬ 
gram  [2]  are  presented. 


The  first  main  result,  an  algorithm  for  solving  parametric  linear 
and  quadratic  programming  problems,  when  the  objective  function  is  non¬ 
linear  in  the  parameter,  is  presented  in  Chapter  IV.  Also  presented  is 
the  class  of  nonlinear  functions  for  which  this  algorithm  is  valid.  The 
finiteness  of  the  algorithm,  including  avoidance  of  cycling  due  to  de¬ 
generacy,  is  then  proven.  The  characteristics  of  the  optimal  solution 
as  a  function  of  the  parameter  are  also  described. 

The  second  result,  an  extension  of  Dantzig's  [9]  formulation  of 
optimal  control  problems  as  generalized  programs,  is  presented  in  Chap¬ 
ter  V.  It  is  shown  that  any  optimal  control  problem  with  the  following 
characteristics  may  be  formulated  as  a  generalized  orogram:  (1)  the 
system  must  initiate  from  some  point  in  a  specified  region  of  the  state 
space;  (2)  the  state  at  the  fixed  terminal  time  can  be  chosen  from  an¬ 
other  convex  region  in  the  state  space  (fixed  initial  and  final  points 
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are  included  in  these  definitions);  (3)  the  state  of  the  system  is  con¬ 
trolled  by  linear  differential  equations;  (4)  the  admissible  control 
region  is  a  convex  polyhedral  set  (for  each  point  in  time)  in  the  con¬ 
trol  space;  (5)  the  loss  criteria  is  a  linear  functional  in  the  state 
and  control  and/or  a  quadratic  functional  in  control  and/or  the  absolute 
value  of  the  control  inputs  (minimum  fuel),  or  the  minimum  time.  It  is 
further  shown  that  these  continuous-time  optimal  control  problems  have 
an  equivalent  generalized  programming  formulation  in  which  the  master 
problem  is  a  linear  program  of  two  or  three  plus  the  dimension  of  the 
state  space.  The  subproblem  to  the  master  program  is  a  parametric  pro¬ 
gramming  problem  of  the  control  space  dimension  and  is  solvable  by  the 
methods  presented  in  Chapter  IV.  This  subproblem  yields  an  extreme  ad- 
missable  control  that,  when  used  with  previously  found  extreme  admis- 
sable  controls,  gives  a  solution  that  is  closer  to  a  feasible  or  an 
optimal  one. 

The  algorithm  and  its  variants  are  presented  in  the  second  part  of 
Chapter  V.  A  flow  chart  of  the  algorithm  is  given,  along  with  a  descrip¬ 
tion  of  each  execution.  Also  included  is  an  initiating  phase  that  ter¬ 
minates  in  a  feasible  solution  of  the  control  problem.  On  completion  of 
the  initiating  phase,  the  algorithm  maintains  a  feasible  control  while 
obtaining  new  controls;  these  new  controls  yield  better  objective  values 
without  disturbing  the  feasibility.  Upper  and  lower  bounds  on  the  op¬ 
timal  objective  value  are  provided  at  each  stage  of  the  algorithm. 

In  Chapter  VI,  the  characteristics  of  the  optimal  controls,  without 
any  additional  assumptions  on  the  system  or  on  the  uniqueness  of  the 
solution,  are  presented.  Also  included  are  the  relationships  between 
the  necessary  conditions  of  Pontryagin  and  the  generalized  programming 
results.  Between  these  optimization  conditions,  a  link  exists  in  the 
dual  variables  of  the  generalized  program  and  the  adjoint  variables 
associated  with  the  optimal  control  problem. 

To  clarify  the  algorithm  and  indicate  its  computational  feasibility, 
a  minimum  fuel  problem  and  a  minimum  time  problem  are  solved  in  detail 
in  Chapter  VII.  The  convergence  properties  and  solution  procedures  are 
illustrated  with  data  obtained  from  computer  runs. 
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Chapter  II 


OPTIMAL  CONTROL 


This  section  defines  an  optimal  control  problem  and  Pontryagin's 
necessary  conditions  for  optimality.  The  emphasis  is  on  those  linear 
systems  for  which  generalized  programming  equivalents  can  be  formulated. 


A.  Definition  of  Dynamic  Control  Systems 


The  basic  control  problem  can  be  described  by  the  differential 
equations: 


dx . 


x.  =  -57-  =  f.  (x . x  ,  u  ,  ....  u  ,  t) 

x  dt  1  l  n  1  m 


(2.1) 


i  =  1,  2,  .  .  .  ,  n  , 


where 


x(t)  = 


x1(t) 


x  (t) 
n 


(2.2) 


is  the  vector  of  state  variables  or  phase  coordinates  which  describe  the 
trajectory  of  the  system  in  Euclidean  space  through  time.  The  control 
function  is  the  vector  of  control  inputs 


u(t)  = 


u^t)' 


u  ( t) 
m 


(2.  3) 


which  influence  the  state  through  the  differential  equations.  The  sys¬ 


tem  at  some  initial  time,  t 


satisfies  the  initial  conditions, 
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x(t  )  £  S  c  En  .  (2.  4) 

o  o 

Thus  the  system  may  have  an  initial  point  x(tQ)  at  any  one  o;f  the  points 
ir.  the  set,  S^.  At  a  terminal  time,  T,  the  system  is  required  to  lie 
in  some  region,  i.e,, 


x(T)  e  StQ  En  .  (2.  5) 

The  time  T  may  be  free  or  fixed,  and  the  sets,  and  S^,  may  be 

fixed  points. 

B.  Admissible  and  Feasible  Controls  and  Reachable  Sets 

The  vector  control  function,  u(t)  must  be  specified  at  every  t 
and  is  required  to  lie  in  an  admissible  control  region,  U^,  where 

u(t)  e  Ut  c  Em  ,  Vt  .  (2.6) 


Definition  2. 1.  An  admissible  control  is  any  vector  function,  u(t), 
for  which 


u(t)  e  c  Em  ,  Vt  e  [ 0 , T]  , 


where  [0,Tl  denotes  the  time  interval  (t|0  <  t  <  T]  . 


The  objective  of  the  control  problem  is  to  find  an  admissible  con¬ 
trol  function  that  transfers  the  state  from  some  point  at  t  to  an- 

o 

other  point  at  T,  while  minimizing 


o 


V 


u  ,  t)  dt  . 
m 


(2.  7) 
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It  is  convenient  at  this  point  to  define  another  variable  » 


x  (t) 
o 


=  f  Vxi .  v  ui . u, 


t)  dt  , 


(2.8) 


and  to  let 


x  ( t)" 
o 


x(  t)  = 


(2.9) 


Thus, 


x  =  f  (x,u.t)  ,  x  (t  )  =  0  .  and  J  =  x  (T) 
o  o  o  o  o 


(2.10) 


Definition  2. 2.  The  reachable  set,  denoted  by  RT,  consists  of  a  set 


of  terminal  x(T)  of  admissible  solutions  to  the  control  problem, 


without  the  condition  x(T)  e  S^. 


R  =  { x  e  E  I x  =  x(T)  , 
T  1  1 


where  x(T)  is  a  solution  to  (2.1)  at  t  =  T  with 


x(t  )  e  S  ,  u(t)  e  U  ,  Vt) 
o  o  t 


Note  that  for  the  fixed  final  time,  T,  if 


sT  n  rt  =  0  , 


there  is  no  admissible  control  to  transfer  the  system  from  an  initial 


point  in  to  a  point  in  S^. 
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Definition  2.3.  A  control  function,  u(t),  defined  for  t  t  [t  ,  Tj  is 

— -  Q 


a  feasible  control  for  the  optimal  control  problem  if  it  is  an  ad¬ 


missible  control  and  transfers  the  system  from  some  state  x(t  )  t  S 

o  o 


to  a  state  x(T)  t  while  x(t)  satisfies  (2.1).  Note  that  a 


feasible  control  exists  iff 


sT  n  rt  *  p  . 


In  the  optimal  control  problem,  we  are  searching  for  a  control 


function,  among  all  feasible  controls,  that  results  in  a  minimal  value 


Assumption  2.  1.  We  will  now  restrict  our  attention  to  functions  f., 


for  i  -  0 . n,  which  are  autonomous,  i.e.,  they  do  not  depend 


explicitly  on  time.  We  will  also  assume  that  the  f^  functions  for 


i  =  0,  1,  . .  are  continuous  in  both  x  and  u  and  are  con¬ 


tinuously  differentiable  with  respect  to  x 


C.  The  Adjoint  System  and  the  Hamiltonian 


for  any  given  u  or  x(o),  let  x  =  x(t)  be  determined  by 


=  f^Cx.u)  i  =  0,  1,  .  ..,  n. 


For  this  choise  of  u,  x(o),  and  the  resulting  x(t),  we  define  the 


ad.joint  system,  ¥  ,  T . T  ,  by 

u  l  n 


d'fi  iL  ^fk(x,u) 

^i  “  dt  ^  5x~ 


—  fk 


i  =0,  1,  .  .  .  ,  n  , 


(2.  11) 


where  the  partials  are  evaluated  at  the  above  x(t),  u(t).  The  solution 


to  (2.11)  is  related  to  the  choice  of  control,  u(t). 
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f\< 

fey#; 


m 


The  Hamiltonian  is  defined  as 


H(Y,  x,u )  =  \Jr'  f  (x,  u)  , 
where  T 1  is  the  transpose  of 


¥ 


o 

1 


T  = 


and 


f(x,u)  = 


f  (x,u) 
o 

f1(x, u) 


f  (x,u) 
n 


Thus  (2.1)  and  (2.11)  become 


c)H 

xi  =  ■  5?: 


(2. la) 


T. 

l 


(2.11a) 


D.  Pontryagin's  Conditions  for  Optimality 

When  the  initial  and  final  points,  x(tQ)  ancJ  >  are  not 

fixed,  the  regions  S  and  S  are  assumed  to  be  smooth  manifolds  or 

o  T 

convex  sets.  A  necessary  condition  for  optimality  in  this  case  is  that 
the  solution  to  (2.1)  and  (2.11)  satisfy  a  transversality  condition. 
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Let  x(i  )  t  S  and  x(T)  t  be  given  points  on  the  boundary  of  S 
o  o  T  J  o 

and  S  :  and  let  D  and  D  be  tangent  planes  of  S  and  S  at 
T  o  T  o  T 

these  points.  Then  the  solution  satisfying  (2.1)  and  (2.11)  will  also 

satisfy  the  transversality  condition,  if  T(t  )  and  f(T)  are  the 

o 

directions  of  the  supporting  hyperplanes,  D  and  D  ,  of  S  and 

o  T  o 

S^,  at  x '  tQ)  and  x(T),  respectively. 

Necessary  Conditions  for  t  <  t  <  T.  Let  u(t)  be  a  feas¬ 
ible  control  with  a  corresponding  trajectory  x(t).  For  u(t)  =  u*(t) 
to  yield  an  optimal  solution  to  the  control  problem,  it  is  necessary  to 
have  a  non-zero  continuous  vector  function  f(t)  corresponding  to  x(t) 
and  u*(t),  (2.1)  and  (2.11),  aad  satisfying  the  transversality  condi¬ 

tions  so  that 

(1)  For  t  e  [t  , T], 

o 

H[x( t)  ,  u*(t)  ,  ?(t)]  =  u(f)‘^u  H[x( t)  ,  u(t),  f(t)] 
and 

(2)  T  (T)  <  0  . 

o  — 

E.  The  Linear  System  and  Control  Constraints 

A  linear  system  is  defined  as  a  dynamic  system  in  which  the 

f,(.x  ,  ...,  x  ,  u  ,  .  ..,  u  )  are  linear  in  x  and  u  for  i  =  1,  ..., 
l  1  n  1  m 

n.  Note  that  f  (x,u)  need  not  be  linear.  This  linear  system  can  be 
o 

described  by  two  matrices,  F  and  G,  as 

x  ( t )  =  Fx(t)  +  Gu(t)  ,  x  ( t )  e  E°  ,  u(t)  e  e"1  ,  (2.12) 

where  F  is  an  n  X  n  real  matrix  and  G  is  an  n  X  m  real  matrix. 
The  linear  system  has  a  fundamental  matrix  [4]  e^^1  ^  that  has 


the  property  of  transforming  x(t)  by: 


M 


when  u(t)  =  0  for  t  t  [ ,  t  3 .  This  fundamental  matrix  arises  from 
the  solution  of  the  differential  equations  in  (2.12)  when  u(t)  =  0. 

The  solution  for  any  function  u(t)  is 


MVV 

x(t  )  =  e  x(tQ)  + 


t  F(t  -T) 


f1  e  1 

-4 


Gu(t)  dt 


(2.  13) 


When  u(t)  e  (J  for  all  t  and  x(t  )  e  S  ,  the  right-hand  side 
t  0  o 

of  (2.13)  determines  a  point  in  the  reachable  set  of  U  ,  S  ,  and  time 

t  o 

t  .  Hence  we  can  state,  for  linear  systems, 


=  <jx  e  E  |  x  =  x(  t^)  , 


F(t  -tj 


x(t^)  =  e 


.t.  F(t  -T) 


1  o'  fl  1 

x(t0)  +  J  e 


Gu ( T )  dt  , 


u(t)  e  U  ,  i  e  [t  , t  ]  ,  x ( t  )  e  S 

t  U  1  Uo 


Throughout  this  paper,  we  will  consider  problems  where  = 

U  c  Em,  i.e.,  the  admissible  control  set  is  constant  over  time.  We 
also  assume  that  U  is  a  bounded  convex  polyhedral  set,  i.e.,  it  is 
bounded  by  hyperplanes  in  m-dimen.'iona  1  space.  Note  that  any  convex- 
polyhedral  set  can  be  expressed  by 


U  =  (  u  e  E  I  Au  <  b) 


for  some  real  fixed  matrix  A  of  dimension  q  X  m  and  for  some  real 
vector  b  of  dimension  q. 
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In  the  following,  we  will  permit  the  initial  and  terminal  sets,  S 

and  ST  c  E*1,  to  be  convex  sets. 

Note  that  f.(x,u)  =  f'.x  +  g'u,  where  f 
1  1  i  1 

vector  and  g.  is  an  m-dimensional  vector.  f. 

l  l 

rows  of  the  F  and  G  matrices,  respectively. 

F.  Loss  Functionals 

In  this  section,  we  will  describe  the  different  classes  of  loss 
functionals.  These  loss  functionals,  when  combined  with  linear  systems 
and  the  above  restrictions,  can  be  solved  by  mathematical  programming 
techniques  that  are  developed  and  discussed  in  the  next  two  chapters, 

Case  1.  Linear  Loss  Functionals. 

We  define  the  linear  loss  case  as  one  that  includes  all  loss  func¬ 
tionals  of  the  form 


is  an  n-dimensional 
and  g^  are  the  i^*1 


f  (x,u)  =  f'x  +  g'u  , 
o  o  o 

where  f  and  g  are  any  real  n  and  m  component  vectors,  respec- 
o  o 

tively.  Thus,  we  can  define  linear  systems  with  linear  loss  functionals 
as  completely  linear  systems. 

Case  2.  Minimum  Fuel  Problems. 

A  certain  well-known  minimum  fuel  problem  is  characterized  by  loss 
functionals  of  the  form 


m 

fU)  =  X  lUJ  ' 

i=l 

Case  3.  Quadratic  Loss  in  Control. 

We  consider  a  function  a  quadratic  only  in  the  control  vector, 

i.  e.  , 


where  Q  is  a  positive  semidefinite  matrix. 


When  a  linear  functional  is  added  to  f(u)  and  modifications  of 


Cases  2  and  3  are  permitted,  the  three  cases  are: 


f  (x,u)  =  f'x  +  g'u  j-  f(u)  , 
o  o  o 


0  ,  Case  1 


where  f(u)  =/z|uJ  ,  Case  2 


u'Qu  ,  Case  3 


(If  f  (x,u)  =  f(u),  then  f  =0,  and  g  =0.)  Thus  the  control 
o  o  o 


problem  can  be  stated  as 


Minimize  x  (T)  , 
o 


x  =  Fx(t)  +  Gu(t)  +  f(u)  U  , 

o 


where 
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x( t  )  €  S  ,  X  ( t  )  =  0 
o  o  o  o 


x(T)  e  ST  ,  and 


u(t)  e  U  =  ( u  e  E  I  Au  <  b} 


The  minimum  time  problem  is  also  considered  where  T  is  not  fixed, 

S  and  are  fixed  points,  and  x  (T)  =  T  -  t  . 

o  T  o  o 

For  linear  systems  described  by  matrices  F  and  G  and  a  given 

polyhedron,  U,  Pontryagin  defines  a  "general  position  condition, "  This 

n 1 

condition  is  satisfied  when  the  vectors  Gw,  FGw,  .  .  .  ,  F  Gw  are  lin¬ 
early  independent  in  En  when  w  has  the  direction  of  one  of  the  edges 
of  U.  For  such  systems,  at  each  point  of  time,  t,  the  function 
T(t)'Gu(t)  achieves  its  maximum  at  only  one  vertex  of  U,  except  on  a 
set  of  measure  zero. 

Before  proceeding  further  with  the  development  of  an  algorithm  to 
solve  these  continuous-time  control  problems,  some  of  the  existing  tech¬ 
niques  used  in  solution  procedures  should  be  mentioned  briefly.  Three 
of  these  techniques  are  mentioned  here. 

Direct  Methods  [5],  In  these  methods,  admissible  and,  if 
possible,  feasible  controls  are  chosen  to  start.  The  gradient  of  the 
cost  functional  (or,  if  the  starting  control  is  not  feasible,  a 
Lagrangian  form  that  takes  feasibility  into  account),  with  respect  to 
the  control  function,  is  determined.  Then,  by  using  gradient  or  steepest 
descent  methods,  a  new  control  function  is  chosen  to  improve  the  ccsc 
functional  (or  Lagrangian)  . 

Indirect  Methods  [6].  Indirect  methods  primarily  seek  solu¬ 
tions  to  the  necessary  conditions  for  optimality.  Some  methods  use 
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arbitrary  initial  or  final  conditions  for  the  adjoint  variable.  In  this 

case,  the  differential  equations,  (2.1)  and  (2.11),  are  in:  grated  to 

find  solutions  for  x(tQ);  during  this  procedure,  a  solut^  'i  to  the 

necessary  conditions  is  retained,  if  possible.  If  x(t  )  is  not  equal 

o 

to  the  original  (known)  x*( t  ) ,  the  gradient  of  some  cost  functional, 

based  on  the  distance  from  x(t  )  to  x*(t  ),  is  used  to  determine  a 

o  o 

new  guess  for  the  final  time  adjoint  variable  values. 

Discrete  Approximations  [7,8].  Mathematical  programming  tech¬ 
niques,  e.g.  ,  linear  programming  or  gradient  projection  methods,  are 
usually  applied  to  a  discrete  approximation  of  the  continuous-time  prob¬ 
lem.  In  these  approximations,  the  system  is  considered  at  a  prescribed 

set  of  instants  in  the  interval  [t  ,  T] .  Only  at  these  times  are  the 

o 

control  inputs  allowed  to  change.  The  differential  equations  are  then 
approximated  by  difference  equations  for  each  time  considered.  Math¬ 
ematical  programming  techniques  are  then  used  to  solve  the  approximation. 

Each  of  the  three  techniques  mentioned  have  their  disadvantages. 

The  direct  methods'  disadvantage  is  that  a  feasible  control  must  be  pro¬ 
vided  initially.  If  not,  the  convergence  methods  cannot  be  guaranteed  to 
terminate  with  a  feasible  solution.  Also,  the  efficiency  of  convergence 
is  highly  dependent  on  the  initial  guess.  The  indirect  methods  also  have 
a  disadvantage  in  that  they  do  not  provide  a  feasible  solution  until  the 
final  step.  At  times,  the  determination  of  a  feasible  solution  is  the 
major  problem  in  optimal  control.  The  basic  disadvantage  of  discrete 
approximations  stems  from  the  large  number  of  variables  or  equations 
introduced  by  the  approximation  process. 

The  methods  developed  in  this  work  combine  the  features  of  both  the 
direct  ana  indirect  methods  and  use  admissible  controls  to  find  a  fea¬ 
sible  solution.  This  combination  continuously  reduces  the  cost  while  it 
retains  the  feasibility  and  converges  on  the  optimum  values  of  the  ad¬ 
joint  variables.  Thus  the  problem,  at  any  iteration  in  the  optimization 
phase,  has  a  feasible  solution  available,  and  the  present  solution  has  a 
measure  of  closeness  to  the  optimum  solution  [9], 
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Chapter  III 

MATHEMATICAL  PROGRAMMING 


In  this  chapter,  the  algorithms  available  for  solving  linear  and 
quadratic  programming  are  reviewed,  and  the  theory  of  generalized  pro¬ 
gramming  is  described.  The  choice  of  the  simplex  method  for  linear 
programming  problems  and  the  complementary  pivot  theory  for  quadratic- 
programming  problems  is  dictated  by  the  ease  encountered  in  using  the 
parametric  programming  methods  presented  in  Chapter  IV. 

It  should  be  noted  that  any  bounded  convex  polyhedral  set  can  be 
represented  (possibly  after  a  change  of  variable)  by  the  set 
X  =  (x|Ax  <  b,  x  >  0]  for  some  real  matrix,  A  and  for  some  real 
vector,  b. 


A.  Linear  Programming 


The  standard  linear  programming  problem  can  be  stated  as 


minimize  z  =  c  x 
subject  to  Ax  <  b  and 
x  >  0  , 


(3.1) 


b  is  a  speci- 
matrix. 


where  x  e  E  ,  c  is  a  specified  n-dimensional  vector, 
fied  m-dimensional  vector,  and  A  is  a  given  (m  X  n) 

Since  minimizing  c'x  is  equivalent  to  maximizing  (-c')x,  only 
minimization  problems  are  discussed.  Hence,  problem  (3.1)  seeks  the 
minimum  of  a  linear  (convex  and  concave)  function  over  a  convex  poly¬ 
hedral  constraint  set;  if  the  latter  is  nonempty,  a  solution  exists 
and  is  known  to  be  at  an  extreme  point  in  the  constraint  set.  Thus  we 
need  only  consider  basic  solutions  to  problem  (3.1),  i.e.,  solutions  in 
which  no  more  than  m  components  of  the  vector  x  are  positive  and 
whose  column  coefficients  are  linearly  independent  in  rows  where  Ax  <  b 
is  satisfied  with  equality. 


15 
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The  dual  problem  to  (3.1)  can  be  expressed  by 

minimize  v  =  by 

subject  to  A'y  >  c  (3.2) 

y  >  0  ,  y  e  Em  . 

The  duality  theorem  of  linear  programming  can  be  summarized  in  two 
statements: 

(1)  for  any  feasible  x, y  [satisfying  the  constraints  of  (3.1) 
and  to. 2) 1 , 


c  '  x  >  b'  y  ,  and 

(2)  for  the  optimal  x*,  y*  of  (3.1)  and  (3.2), 

c '  x* 

(Ax*  -  b)  1  y* 

(A'y*  -  c)  '  x* 

If  the  x  vector  is  augmented  by  m  components  to  include  slack  vari¬ 
ables  and  the  matrix  A  is  augmented  by  I,  the  constraint  inequalities 
are  equivalent  to 


=  b’y* 

*°l 
= 0  \ 


complementary  slackness  conditions  . 


Ax  =  b 
x  >  0  , 


(3.3) 


where  A  and  x  are  now  the  augmented  matrix  and  vector,  respectively. 

Since  we  need  only  investigate  the  extreme  points  of  the  constraint 
set,  we  need  only  allow  basic  solutions  corresponding  to  choosing  m 
linearly  independent  columns  of  A, and  the  components  of  the  vector  x 
corresponding  to  the  m  columns  of  A.  The  m  columns  of  the  augmented 
A  form  a  nonsingular  matrix  B,  called  the  basis  matrix.  The  corre- 
SDonding  components  of  x  are  called  the  basic  variables.  Hence,  a 
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basic  feasible  solution  to  (3.1)  is  one  in  which  the  values  of  the  basic 


variables  are  nonnegative,  and  all  the  other  variables,  called  the  non- 


basic  variables,  are  at  value  zero.  Let  x  represent  the  vector  of  the 

B 


basic  variables  corresponding  to  B.  Then  the  oasic  solution  to  the 


linear  equations  in  (3.3)  is 


xB  =  B  b  , 


x .  =  0  , 


where  i  is  nonbasic.  This  is  a  basic  feasible  solution,  provided 


XB  —  °* 


The  Simplex  Method.  The  simplex  method  is  reviewed  in  detail, 


since  a  variation  of  it  is  employed  in  Chapter  IV  for  the  parametric 


programming  procedures.  This  method  is  presented  in  matrix  form.  Here, 


the  linear  program 


minimize  z  =  c'x 


subject  to  Ax  =  b 


(3.4) 


x  >  0 


is  observed,  and  the  augmented  system  of  equations 


1  -c'  z 


0  A  x 


(3.5) 


is  used. 


Given  any  basis,  B,  let  the  augmented  basis  be  B,  where 


1  ~°B 


0  B 
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and  rewrite  (3.5)  as 


—  — 

-  - 

—  — 

1  -c '  -c ' 

z 

0 

B  R 

x^ 

— 

B 

0  B  R 

b 

—  — 

L  Rj 

where 


A  = 


and 


c 


B 


c  = 


(3.6) 


CB  is  the  vector  the  components  of  c  corresponding  to  the  basic 

variables  x  . 

B 

Since  B  is  nonsing  lar,  B  is  also  nonsingular; 


_-l 

B 


cbb 


-1 


0  B 


-1 


Multiplying  (3.6)  by  B  1  and  then  rearranging  it, 


z 

1 

0 

-c'+c^B-1R 

_XB_ 

0 

b 

B_1R 
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we  get 


(3.7) 


$ 


By  setting  the  ncnbasic  variables,  x  ,  at  level  zero, 

R 


1  cbb 


0  B 


CBD"lb 


B-1b 


where  xQ  =  B  b. 

If  B  is  a  nonnegative  vector,  the  basis  B  is  feasible, 

and  thus  the  current  solution  is  a  basic  feasible  solution. 

Look  at  any  variable  with  a  corresponding  column  A. 

and  a  cost  coefficient  c.;  this  variable's  column  in  the  transformed 

J 

system  of  (3.7)  is 


-c  ,+c’B  ^A  . 
J  B  j 


-1  th 

If  x  is  a  basic  variable,  BA  is  the  r  unit  vector,  if  A. 

J  th  3  3 

is  the  r  column  of  B.  (Note  that,  in  this  case,  c^  would  be  the 

rtB  component  of  c  ,  and  x.  would  be  the  rtB  component  of  x  . ) 

B  j  B 

Thus  the  first  component  of  (3.8)  becomes 


-c  .  +  c' B  *A  .  = 
J  B  j 


-c  .  +  c  .  =  0  ; 
J  J 


moreover  (3.8)  is  a  unit  vector. 


m 


Proposition  3.1.  If  all  c.  -  c'B  A .  >  0,  the  current  basis  B  is 
- - -  J  B  J  - 

optimal. 


Proof  of  Proposition  3.1. 

Assume  6  =  c  -  c'B  1A.  >  J,  V.;  then,  from  (3.7)  and  (3.8), 
j  J  B  j  -  j 
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:  B  b  +  y  b.x. 
15  L  J  J 


Note  that  =  0  for  j  corresponding  to  a  basic  variable.  Thus  any 


change  from  the  current  solution  would  result  in  an  increase  of  some 


x.  (nonbasic),  and  the  value  of  z  would  increase  or  remain  unchanged. 


Hence  no  improvement  in  the  objective  is  obtained  with  any  other 


solution. 


Q.  E.  D. 


m  1,  we  have  an  optimality  condition  for  any 


feasible  basis; 


CBD~lAj  >  0  , 


(3.  9) 


If,  on  the  other  hand,  the  left-hand  side  of  (3.9)  was  strictly  negative, 


for  some  j  =  s,  then  increasing  x  and  adjusting  the  values  of  the 


basic  variables  until  one  dropped  to  value  zero  (thus  replacing  a  current 
basic  variable)  would  decrease  the  objective  function,  provided  x 


entered  at  a  positive  level.  The  simplex  method  changes  the  basic  set 


at  each  iteration  with  the  entering  variable,  x  ,  designated  the  non- 

s 


basic  variable  with  the  most  negative  relative  cost  factor,  t>  .  The 


exiting  variable  is  the  first  basic  variable  to  be  driven  to  zero  as 


the  entering  variable  increases  above  zero  (assuming  nondegeneracy  and 
bounded  solutions).  The  method  terminates  with  the  current  basis  being 


optimal,  when  (3.9)  is  satisfied  for  all  variables. 


When  the  variable  x  is  chosen  as  the  entering  variable,  the 

s 


exiting  variable  can  be  determined  by  examining  the  ratios 


(B~lb)i 


for  all  i  , 


(3. 10) 


re  1 A  ^  _  >  0.  From  (3.7),  the  current  basic  variables  are  ex¬ 


pressed  as 
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is  tho  one 


Thus  the  first  variable  driven  to  zero  in  the  vector  xR 

corresponding  to  the  minimum  of  the  ratios  defined  by  (3.10). 

The  simplex  method  can  be  carried  out  in  two  ways.  The  first 

way  (called  the  revised  simplex  method)  is  to  substitute  A  which  cor- 

s 

responds  to  the  entering  variable  x  for  A  which  corresponds  to  the 

s  r 

exiting  variable  x  in  the  basis  B.  With  this  substitution  both  the 
new  solution  and  the  relative  cost  factors  can  then  be  calculated.  The 
second  way  is  to  pivot  in  the  augmented  matrix 


about  the  term  ^ B  1Rsj  ,  where  s  corresponds  to  the  entering  vari¬ 
able  and  r  corresponds  to  the  exiting  variable.  The  pivoting  opera¬ 
tions  do  not  change  the  canonical  form  of  the  basic  variables  which 
remain  basic,  but  they  do  force  the  column 


to  the  canonical  form  of 


where  e  is  the  unit  vector  with  a  one  in  the  r  component;  this 
r 

will  alter  all  of  the  other  columns  corresponding  to  the  nonbasic 


variables.  Note  that  once  a  feasible  basis  is  determined,  the  simplex 
method  insures  that  all  succeeding  bases  are  feasible. 


To  obtain  an  initial  feasible  basis,  phase  I  of  the  simplex  method 

adds  artificial  variables  to  (3.4)  and  solves  a  new  linear  program.  Let 

E  be  an  m  X  m  matrix  with  only  diagonal  terms,  and  let  e  =  +1,  if 

11 

b  >  0,  and  e^  =  -1,  if  b^  <  0;  then,  the  new  linear  program  is 


i=l 

Ax  +  Ev  =  b 

x  >  0,  v  >  0  ,  (3. 11) 


and  the  solution  terminates  in  a  basic  feasible  solution  to  (3.4),  when 
the  simplex  method  is  applied  to  (3.11).  The  optimal  value  of  z  in 
(3.11)  is  zero  iff  (3.4)  is  feasible. 


v 


D.  Quadratic  Programming 

The  standard  quadratic  programming  [3]  problem  can  be  stated  as 

minimize  z  =  c'x  +  x'Qx 
subject  to  Ax  >  b 

x  >  0,  (3. 12) 

where  x  e  E  ;  c  is  a  specified  n-dimensional  vector;  b  is  a  speci¬ 
fied  m-dimensional  vfector;  A  is  a  specified  (m  X  n)  matrix;  and  Q 
is  a  specified  (n  x  n)  matrix.  It  is  hereby  assumed  that  Q  is  posi¬ 
tive  semidefinite. 

Since  problem  (3.12)  is  a  convex  programming  problem,  the  Kuhn- 
Tucker  necessary  conditions  are  also  sufficient  conditions  for  optimality. 
Thus  a  solution,  x,  to  the  following  necessary  conditions  is  an  optimal 
solution  to  (3.12). 
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u  =  c  +  2Qx  -  A ' y  >  0 


v  =-b  +  Ax 


>  0 


>  0 


>  0 


xiui  =  °-  Vi  -  °* 


Vi 


'3.  13) 


If  we  define 


u 

X 

W  - 

,  z  = 

_  V  _ 

_y_ 

2Q 

-A'“ 

c 

M  = 

,  and  q  = 

_  A 

0  _ 

_  "b_ 

the  necessary  conditions  may  be  written  as 


w  =  Mz  +  q 


w,  z  >  0, 


w. z .  =  0. 
i  l 


(3.14) 


where  M  is  p  X  p. 

Complementary  Pivot  Theory.  Problem  (3.14)  is  a  statement  of 
the  fundamental  problem  of  the  complementary  pivot  theory  [3].  Although 
(3.14)  is  solvable  by  this  theory  for  various  classes  of  M,  the  dis¬ 
cussion  here  will  be  restricted  to  M  being  positive  semidefinite,  as 
it  is  in  the  quadratic  programming  problem  (3.12). 

Note  that  we  are  looking  for  a  complementary  solution  to  the 
linear  equations  in  (3.14),  i.e.,  a  solution  to 


w  =  Mz  +  q  , 

23 
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with  w^z^  =  0,  Vi  •  We  will  initiate  with  a  solution  that  is  comple¬ 
mentary  but  that  may  not  be  nonnegative.  We  will  then  retain  this 
complementary  property  while  seeking  a  nonnegative  solution. 

The  problem  in  the  structured  form  of 


(3. 15) 


is  observed  with  the  transformations  being  made  by  substituting  a  vari¬ 
able  z.  (or,  in  later  steps,  some  w.)  in  the  extreme  left  column, 

l  l 

replacing  a  variable  in  the  column,  and  then  pivoting  on  the  system  of 
equations  by  changing  the  column  q  and  the  matrix  M.  The  variables 
in  the  left  column  are  called  basic,  and  the  variables  in  the  row  above 
the  matrix  M  (or  M  after  transformation)  are  called  nonbasic .  The 
problem  is  initiated  by  setting  =  q^  and  z.  =  0  for  all  i.  If 
any  q^^  is  negative,  pick  the  w  corresponding  to  min  q^  and  let  it 
be  a  distinguished  variable.  The  following  can  be  taken  as  a  general 
iteration. 

Increase  the  complement  [defined  by  (3.14)]  of  the  distinguished 
variable  and  determine  the  blocking  variable  which  is  either 

(a)  a  basic  variable  being  driven  below  its  lower  bound  (usually 
zero)  by  an  increase  of  the  driving  variable,  or 

(b)  the  distinguished  variable  which  is  driven  toward  zero. 

[The  first  variable  to  block  in  either  (a)  or  (b)  becomes  the 
blocking  variable. ] 

If  the  blocking  variable  is  not  the  distinguished  variable, 

then  replace  the  basic  blocking  variable  with  the  increasing  nonbasic 

(driving)  variable  by  pivoting  about  the  point  m  in  the  matrix  M, 

r  s 

where  m  is  the  term  in  the  current  matrix  that  corresponds  to  the 

th  rS  th 

s  column  (the  driving  variable)  and  the  r  row  (the  blocking  vari¬ 


able)  .  Now  increase  the  complement  of  the  former  blocking  variable 
(now  nonbasic)  until  a  new  blocking  variable  is  found. 


If  the  blocking  variable  at  any  stage  is  the  distinguished 


c. 


Generalized  Programming 


The  general,  'ed  programming  problem  can  be  represented  by 


Choose  a  vector,  P,  in  a  convex  set,  C  c  En,  such  that 


maximize  A 


subject  to  U  A  +  Pi,  =  S 
0  M 

M  =  1. 


(3. 16) 


where  UQ  and  S  are  specified  n-dimensional  vectors,  and  p  is  a 
scalar.  [The  results  here  are  easily  applied  to  an  extended  form  of 
(3.16),  where  the  linear  equations  become 

V  +  V 1  +  P2m2  +  •  •  •  +  PqMq  =  S 

=  1  >  Vi  , 


and  eacii  P.  is  drawn  from  a  convex  set  C..] 

Thus,  we  are  looking  for  some  vector  P*  or  a  convex  combination 
i  * 

of  vector  P  ,  all  in  set  C,  so  that  the  linear  equations  are  fea¬ 
sible,  i.e., 


UqA  +  P  =  s 


(3.  17) 


U  A  +  y 

n  / 


i* 

p  u,  =  s 


2-i- 


>  o  , 


(3. 18) 


and  t he  resulting  value  of  A  is  a  maximum  over  the  choice  of  all  the 
elements  in  set  C,  which  satisfy  the  linear  equations.  Note  that,  if 
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any  set  of  P  is  in  C,  any  convex  combination  of  that  set  is  also  in 
C.  Hence  (3.17)  and  (3.18)  are  equivalent  when 


P 


* 


Mi  >  0  . 


The  solution  procedure  assumes  we  have  on  hand,  initially,  n 
particular  choices  of  P1  e  C  so  that  the  following  linear  program 
(called  a  restricted  master) 


max 
M 

subject 

Ml  +  .  .  .  +  Mn  =  1 

M.  >  0  (3.19) 


A 

-i 

to  u^A  *+-  p  it  +  . , ,  +  p  n  =  s 

0  H1  Mn 


has  a  unique,  feasible,  nondegerate  solution  with  the  basis  being 
defined  as 


B 


0 


0 


1  .  .  .  1 


i  0 

and  being  nonsingular  (by  definition) .  Since  for  each  P  e  C,  P  = 


2 


p^p^,  where  p*?  is  a  solution  to  (3.19),  is  in  C  and  is  a 


l  l 

feasible  solution  to  (3.16),  but  not  necessarily  the  optimal  solution. 

To  test  P°  [and  hence,  any  solution  to  (3.16),  generated  from  a 

0 

basis]  for  optimality,  a  row  vector  it  =  if  is  determined  to  satisfy 


jT°B°  =  (1 ,0,  ...  ,0) 


(3. 20) 
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we  find  a  vector  P 


,  which  is  not  necessarily  unique,  and 


'•''•din 


n  +  1 


a  value  c  so  that 


_0_ n+1 

O  =  I!  P 


min  — 0— 
PcC  11  P  ’ 


(3. 21) 


where  P  = 


If  o=0,  the  current  solution  is  an  optimal  one. 

n+1 


Ii  o<0,  (3.19)  is  augmented  by  P  and  the  new  linear  program  is 

then  solved.  The  general  iteration  starts  with  a  solution  to  the  re¬ 
stricted  master  program 


maximize 


n+k 

V 


subject  to  UqA  +  )  P1p.  =  S 


0  . 


(3.  22) 


Let  B*v  be  the  optimal  basis  to  the  linear  program  (3.22),  and  let  lTk, 
the  dual  (optimal)  variable  to  (3.22),  be  defined  analogous  to  (3.20). 
Then,  and  pn+l'+1  are  found  from  the  subproblem, 


, .  .  1<+1  — k-n+k+1 

find  b  =  rt  P 


min  _k— 
„  „  it  P 
PeC 


(3. 23) 


k+1  th 

If  o  =0,  tlie  solution  to  the  k  iteration  of  the  master 

problem  is  optimal.  If  b  <  0,  then  pn+^+l  can  ac]j0fnec)  fG 


(3.22),  and  1  he  solution  to  (3.16)  is  improved.  The  value 


k+1 


is 


k 


he  maximum  amount  by  which  the  value  of  the  current  basis  A  can  be 
improved.  Thus,  A  -  c  constitutes  an  upper  bound  to  the  optimal 

solution  of  (3.16) .  It  is  known  that  these  upper  bound  evaluations  can 
vary  considerably  from  one  interation  to  the  next.  Accordingly,  the 
least  of  these  evaluation:-,  is  saved  from  all  iterations,  including  the 
current  one. 
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It  has  been  shown  that,  if  C  is  bounded  and  the  initial  solution 


_ k  _* 


k*  * 


to  (3.19)  is  nondegenerate  (n.  >0),  n  ->  n  and  P  -»  P  [where 


■  2  ^  • 


and  (i.  is  a  solution  to  (3.22)]  ,  on  a  subsequence  k,  and  that  P  =  P 
1  _* 

is  optimal  for  (3.16).  it  satisfies  the  properties 


_* 

7t  ^  0 


(3.24) 


7F  P  >  n  P  =0,  for  all  P  e  C 


Moreover,  if  C  is  a  polyhedral  set,  then  the  subproblem  (3.23)  is  a 


linear  program,  and  the  iterative  process  terminates  in  a  finite  number 


of  steps.  It  should  be  noted  that,  in  any  case,  the  objective  function 


improves  with  each  iteration,  and  a  feasible  solution  always  exists  to 
the  master  problem.  Also,  the  initial  solution  (or  columns)  for  (3.19) 


can  be  obtained  by  a  procedure  similar  to  a  phase  I  simplex  method. 

Remembering  that  the  usual  form  of  a  generalized  program  includes 


the  sum  of  the  vectors  P.  e  C.,  where  the  C.  are  convex  sets,  the 

li  l 


vector  S  need  not  be  fixed,  but  it  must  be  drawn  from  a  convex  set, 


Thus  the  generalized  program  becomes 


UA  +  Pi 
o  h 


-  Sv  =  0 


v  =  1  , 


(3. 25) 


where  P  e  C  and  S  e  S .  In  this  case,  the  subproblem  is  extended  to 


as  in  (3. 23)  and 
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where  S 


A  =  min  n  S 


s  e  A  , 


If  b  or  A  <  0,  then  the  corresponding  vector  or 


vectors  is  entered  into  the  master  problem.  If  both  6  and  A  =  0, 
the  current  solution  is  optimal. 

The  generalized  programming  problem, 


Primal : 


UqA  +  Pp  =  S 


P  e  C 


(3. 16) 


has  as  its 


Dual : 


find  a  vector  it  so  that 

7f  ^  >  0  ,  VP  e  C 


_  p 

n  ^  =  0  ,  some  P  e  C 


•tl- 


(3. 26) 


This  dual  is  the  equivalent  of  finding  a  particular  hyperplane  to  support 

the  convex  set  C.  If  a  solution  to  the  dual  is  known,  then  a  solution 

_* 

to  the  primal  may  be  found  using  the  dual  solution,  n  ,  to  find  the 
* 

vectors,  P  e  C,  that  satisfy 


=  0  . 


*  * 

If  P  is  unique  and  the  primal  has  a  solution,  P  must  be  the  solu- 
* 

tion.  If  P  is  not  unique  and  the  primal  has  a  solution,  then  some 

* 

convex  combination  of  all  the  P  must  form  the  primal  solution. 
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Chapter  IV 
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PARAMETRIC  PROGRAMMING 

In  this  chapter,  algorithms  are  presented  for  solving  parametric 
linear  and  quadratic  programming  problems,  where  the  dependence  on  the 
parameter  is  nonlinear  and  occurs  only  in  the  linear  part  of  the  objec¬ 
tive  function.  These  parametric  programming  problems  arise  in  the  sub¬ 
problem  of  the  generalized  programming  formulation  of  the  o,  imal  control 
problems. 


A.  Parametric  Linear  Programming 

We  consider  the  following  problem  linear  in  x 

* 

find  x  (t)  to 
minimize  J ( t) ’ x 
subject  to  Ax  =  b 
x  >  0 

t  e  [T^Tg]  ,  (4.1) 


and  the  following  problem  quadratic  in  x 


* 

find  x  (t)  to 
minimize  /(t)'x  +  xQx 
subject  to  Ax  >  b 
x  >  0 

t  e  [T  , T  ]  .  (4.2) 

1  ^ 


In  both  of  the  above  cases,  A  is  a  given  m  X  n  real  matrix,  b 
is  a  given  n-dimensional  vector,  Q  is  an  n  X  n  positive  semidefinite 
matrix,  x  is  a  vector  in  En, 


and 


is  a  given  vector  valued  function,  each  component  of  which  is  a  solution 
to  some  homogeneous,  linear  differential  equation  with  constant  real  co¬ 


efficients  that  may  depend  on  k.  Such  7  (t)  are  of  the  form 

k 


7 


(t) 


I 


\i(t) 


(4.4) 


where  p,  .  ( t) 
ki 

so  that 


is  a  polynomial  with  real  coefficients  of  degree  m 

ki 


N 

i=l 

and  s  .  are  constants  so  tha:,  if  s  j.s  complex  for  i  odd, 
k  1  k  i 

ski  +  1  is  its  conjugate  and  p  (t)  =  Pki+1(0.  It:  follows  then  that 

these  7,  (t)  are  real-valued  functions  of  t. 
k 

The  lemmas  and  theorems  that  follow  are  required  to  show  convergence 
of  the  proposed  algorithm. 

Lemma  4.1.  If  7  (t)  is  a  solution  to  a  homogeneous  linear  differential 
equation  with  constant  real  coefficients  of  order  N  and  if  for 
some  t  =  t 


d  .  . 

=  at 

dN_1[7^(t)] 

* 

'  *  “  ,  N-l 

t. 

t„ 

dt 

=  0 


then  7„(t)  =  0  for  all  t. 


Proof  of  Lemma  4.1. 

7  (t)  solves  an  equation  of  the  form 
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Tn  '*(t)  =  V*(t)  +  ai  -dt-  +  •••  +  Vi  — F 

at  dt 


(4.5) 


At  t  =  tQl 


n 


(4.6) 


By  taking  the  derivative  of  both  sides  of  (4.5)  and  substituting  (4.6), 


N+l  7*(t) 


=  0  . 


If  this  procedure  is  continued,  all  derivatives  of  7*(t)  at  t  =  t 
become  zero.  Therefore,  with  7*(t)  =  0  and  all  of  its  derivatives  at 
zero  for  t  =  tQ  and  with  7*(t)  being  able  to  expand  (at  t  =  tQ)  to 
a  Taylor  series,  7  (t)  must  be  constant  and  have  value  zero  for  all  t. 

Q.E.D. 


Definition  4.1.  A  vector  y  is  said  to  be  lexicographically  greater 

than  zero,  if  at  least  one  component  is  non-zero  and  the  first  such 
component  is  positive;  this  vector  can  be  denoted  as 


y  ^  0 


A  vector  y  is  lexicographically  greater  than  a  ve^..or  z, 


y  ^  z 


if  y  -  z  ?-  0.  A  vector  is  said  to  be  lexicographically  greater  or 
equal  to  zero,  if  it  i ~  lexicographically  greater  than  zero  or  equal 
to  zero. 
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A  similar  definition  is  true  for  one  vector  to  be  lexicographically 
greater  or  equal  to  another  vector. 


Df  ’nition  4. 2.  Let  /  (t)  be  a  real  scalar  function  of  t.  Then  the 
N-component  vector  D  (t)  can  be  defined  by  its  components 

7  * 


D  (t)  .= 


di_17Jt) 


dt 


i-1 


Thus  the  vector  function  D.^,  (t)  is  defined  by  the  function  /^(t) 


and  its  first  N-l  derivatives. 


Lemma  4.2  [10].  If  f(x)  has  a  derivative  at  c  and  f' (c)  >  0,  then 


a  positive  number  6  exists  so  that  for  c  <  x  <  c  +  6,  f(c)  <  f(x) 


Theorem  4.1.  Let  i„(t)  be  a  member  of  the  class  of  solutions  to  ho- 
-  * 


th 


mogeneous,  constant  real  coefficient,  N  order,  linear  differential 


equations,  and  let  (t)  exist  as  it  is  defined  in  Definition  4.2 

Then,  if  D  (t)  =0  or  if  D  ( t„)  0,  a  5  >  0  exists  so  that 

7*  0  7*  0 

>0  when  t  e  [tQ,to  +  5). 


Proof  "f  Theorem  4.1. 


If  D  (t  )  =  0,  then,  according  to  Lemma  4.1,  y  ( t)  =  0  for  all 
7*0  * 


t  and  5  =  00, 

If  Du  (trt)  0,  either  7^(tn)  >0  or  its  lowest  order 


7*  0 


0 


derivative — one  that  is  non-zero  at  t  =  t — is  greater  than  zero.  If 


/  (t  )  >  0,  then,  by  continuity  a  5  >  0  exists  for  7  (t  )  >  0  when 

7T  Q  "*"0  - 

If  7  ( t  )  =  0,  let  the  lowesl 
be  the  derivative,  and  let 


t  t  [tn,tri  +  6).  If  7*(tg)  =  0,  let  the  lowest  order,  non-zero  deriv- 


0’  0 
ative  at  t 


Hero,  /(l)  is  a  vector  whose  1  component,  7 ^ ( t ) ,  is  a  real  scalar 
function.  At  time  t^,  let  be  an  optimal  basis  for  the  linear 

program 


minimize  /  ( t  ) ' x 
subject  to  Ax  =  b 


x  >  0 


(4.  7) 


*  * 

Let  the  solution  be  x  =  x  (t  ).  Since  x  (t  )  is  a  feasible  solution 

0  0 

to  Ax  =  b, x  >  0,  it  remains  a  feasible  one  to  problem  (4.1)  for  all 
t,  but  it  is  not  necessarily  an  optimal  solution.  Thus  how  the  opti¬ 
mality  test  for  varies  as  t  takes  on  the  values  t  =  t  +  e, 

where  e  >  0,  must  be  investigated.  Let 


'k(t'V  -  7k(t)  -  'b  (t)  Bo  \  • 

0 


where  A]  is  the  k  column  of  the  matrix  A, 


(4.8) 


tk  =  inf  (e|/k(t0  +  e,BQ)  <  0]  ,  and 


=  min  t  . 
0  k  k 


(4.9) 


(4. 10) 


It  is  possible  that  the  e  ,  presented  above,  is  zero.  7  ( t ,  B  )  is 

U  K  U 

the  relative  cost  factor  for  any  t  of  column  k  when  B^  is  chosen 

as  the  basis.  Then  the  ordering  of  columns  A  can  be  taken  so  that 

Ah  ,  .  .  .  ,  A  correspond  to  the  m  columns  of  B„. 

1  m  0 

For  any  basic  variable  x^,  associated  with  the  optimal  basis  B^, 


/  .  ( t ,  B  )  =  /  .  ( t )  -  7  (t)  B  A.  =  7.(0  -  7  .  ( t)  -  0,  Vt  . 

1  0  1  Bq  Oi  1  1 


And,  for  1  = 
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WV  >°-  Vk  - 


(4. 11a) 


(4.  lib) 


•  •  •  '  •  I  *  m  •  -  •  •  ^  «  »  *  fc1*  t  r  «  *  ft  »  •  ^  '  t* 


itarxxr.srznim 


U 


with  equality  being  held  when  k  corresponds  to  a  basic  variable. 

Therefore,  the  solution  to  (4.1)  remains  optimal,  i.e.,  it  satisfies 

7k(t.B0)  >  0  when  t  e  [t^t^  +  £0^  for  some  €q  >  °*  Siven  by  (^.lO). 

Let  7  1  ( t , B)  be  the  jth  derivative  (with  respect  to  t)  of  the 

relative  cost  vector  7(t,B)  when  B  is  the  basis  under  consideration. 

Let  7^(t,B)  be  the  component  of  the  above  vector  corresponding  to 
K  1 

column  k  of  A.  Let  A  be  the  new  linear  programming  matrix  ob¬ 
tained  after  deleting  all  columns  (variables)  for  which  the  relative 
—  —0 

cost  factors  =  ^k^O’B0^  are  strictly  positive  for  t  =  tQ 

A  general  iteration  is  given  with  tQ  =  T  ;  a  flow  chart  of  the  algorithm 
follows  the  iteration. 


Step  I:  Solve  the  linear  program 

minimize  7(t  ) 'x 
subject  to  Ax  =  b 

x  >  0  (4.12) 


to  obtain  the  optimal  basis  B^.  If  the  solution  is  unique  at  tQ 
(i.e.,  all  relative  cost  factors  for  nonbasic  variables  are  strictly 
positive),  proceed  to  Step  III. 


If  the  solution  is  non-unique,  fix 
starting  with  j  =  1  and 


t  =  t 


A°  =  A, 


0 


and  proceed  to  Step  II, 


Step  II:  Let  the  matrix  AJ  be  composed  of  the  matrix  Bj_i  and  all 

•1  -1-1 
k  columns  of  A"  having  the  relative  cost  factors  7  (t  ,  B.  )  =  0, 

_ "j  — 1  s  t  ^ 

where  (tg,B  ^)  refers  to  the  (j-1)  derivative.  To  simplify 

notation  in  (4.13)  below,  let  the  new  x  any  7  vectors  corresponding 

to  AJ  also  be  denoted  by  x  and  7,  although  they  are  now  shortened 

x  and  7  vectors.  Then,  solve  the  linear  program 
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minimize 
subject  to 


'  <,0'DJ-1)'X 


Ax  =  b 


x  >  0  . 


(4.13) 


Let  B.  denote  the  optimal  basis.  [Con.put ationally,  it  is  convenient 
J 

to  start  with  the  previously  optimal,  basic  feasible  solution  corre¬ 
sponding  to  j-1  and  then  to  apply  the  simplex  method  to  obtain  an 
optimal  solution  to  (4.13).]  If  the  solution  to  (4.13)  is  unique  or  if 
j  =  N  -  1,  use  the  optimal  basis  and  proceed  to  Step  III. 

If  the  solution  is  non-unique  and  j  <  N  -  1,  increase  j  by  1 
and  repeat  Step  II. 


Step  III:  Using  the  optimal  basis  from  Step  I  or  II  in  place  of  B^  in 
(4.8)  and  (4.9)  for  all  columns  k  (optimal  basic  columns  can  be  ignored 

since  their  e,  =  +oo)  find  e,  .  Then  calculate  The  solution, 

*  _1  k  k  0 
x  (t)  =  B  b,  is  then  optimal  for  all  t  e  [t  ,  t..  +  e  ] .  Moreover,  it 

will  be  shown  that  eQ  >  0. 

If  t  +  e  >  T  ,  the  parametric  programming  problem  is  solved. 

U  U  “*  ^ 

However  if  the  solution  is  not  reached,  repeat  the  general  iteration 
with  t  =  tQ  +  replacing  tQ. 

That  this  algorithm  does  terminate  in  a  finite  number  of  steps  to 
a  solution  of  (4.1)  for  all  t  e  [T  ,T  ]  remains  to  be  shown.  The  re- 

J.  Cj 

mainder  of  this  section  is  devoted  to  showing  a  finite  number  of  steps 
to  the  solution. 


Lemma  4.  3.  If  the  relative  cost  factors  for  some  basic  BQ  are  zero  for 
a  subset  of  columns  S  and  positive  for  the  remaining  columns  T, 
then  the  same  is  true  for  any  basis  B  whose  columns  are  in  S. 

Proof  of  Lemma  4.3. 

The  vector  of  coefficients  of  the  objective  equation  of  the  original 

matrix  can  be  replaced  by  the  relative  cost  vector  for  B  .  The  price 

0  0 
vector  it  of  simplex  multipliers  relative  to  satisfies  it  B^  =  0 
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and  that  of  it  =  0,  since  the  objective  coefficients  for  both 
columns  of  and  B  are  now  zero  by  hypothesis.  Hence  = 

and  it  follows  that  their  relative  cost  factors  are  identical. 

Q.E.E 


Theorem  4.  2.  At  the  end  of  the  general  iteration,  the  following  vector 
relations  are  satisfied: 


VVM 


fi  "k(VB) 

D'VVB)i  - 


Tm  h<(VB) 


for  k  =  1,  . . . ,  n 


DC7,  (t_, B)]  =  0  for  all  k 
k  0  , 


where  B  is  the  final  basis  on  terminating  the  iteration  of 
Step  II  at  tg. 


Proof  of  Theorem  4. 2. 

For  k  corresponding  to  the  basic  variables  of  (4.11), 

DC/,  (t  ,B)]=  0.  For  the  basis  B,  let  the  basic  variables  be  x  , 
k  U  B 

and  let  the  nonbasic  variables  be  x  .  The  problem  is  separated  as 

R 


II 


0 


B  !  R 

X 

B 

L  :  J 

XR 

1 

rxBi 

z  - 

7B<  J 

1 

1 

+  1 

rows  of 

(4.14), 

=  b 


(4.  14) 


=  0  . 


unit  vectors,  and  the  system  becomes 


I  i  R 

i 


=  b 


r~  ,  -1 

B 

L  i  '"J 

X 

-  RJ 

=  z. 


where  7  is  the  relative  cost  factor  vector  of  the  nonbasic  variables. 
R 


R  is  separated  into  two  matrices,  R  and  R 


2’ 


so  that  the  relative 


cost  factors  corresponding  to  the  columns  of  R^  are  zero,  and  those 


corresponding  to  the  columns  of  R  are  negative.  Then,  the  problem 


rows 

1 . m 


xr\ 


XR„ 


=  b 


row 

m+1 


z  - 


\(VB)  *  0 


=  z„ 


I 


.v 

•*i1  ,1 


row 
m+2  ■ 


min  z. 


yl '  (t,B) 

R1 


V 


R, 


=  0 


(4. 15) 


is  observed,  where  only  the  variables  corresponding  to  the  columns  of 
R^  and  I  are  allowed  to  enter  the  basis. 

According  to  Lemma  4.3,  the  pivoting  procedures  of  the  simplex  algor¬ 
ithm  retain  the  zero  elements  of  row  m  +  1  at  level  0  and  the  positive 
elements  of  row  m  +  1  are  at  positive  values  for  every  stage;  these 
procedures  terminate  with  all  relative  cost  factors  of  row  m+2,  corre¬ 
sponding  to  I  and  R  ,  at  nonnegative  values.  At  termination,  because 
of  the  simplex  method  stopping  rule,  a  new  set  of  basic  variables  is  found 
having  the  property  of  the  components  of  the  basic  variables  in  rows  m  +  1 
and  m  +  2  being  at  zero  (after  pivoting);  the  components  of  the  nonbasic 
variables  are  either 

(1)  zero  in  row  m  +  1  and  n'innegative  in  row_  m  +  2,  for  vari¬ 
ables  corresponding  to  columns  of  I  or  R^  in  (4.15)  or 

(2)  strictly  positive  in  row  m  +  1,  for  variables  corresponding 

to  columns  of  R„  . 

2 

If  the  variables  are  as  in  (1)  above,  the  ones  having  zero  components 
in  row  m  +  2  are  chosen  with  their  columns  for  consideration  in  the 
next  stage  of  the  algorithm.  Once  a  nonbasic  variable  has  a  positive 
relative  cost  factor  at  any  stage  j,  it  can  be  assumed  that  its  rela¬ 
tive  cost  factors  were  at  zero  in  previous  stages  also;  .V-once  it  can  no 

longer  enter  the  basis.  Since  the  relative  cost  factors  in  the  first 

t  h 

k  -  1  stages  can  never  change  sign  by  pivoting  in  the  I; "  and  later 
stages,  its  derivative  vector  must  be  lexicographically  greater  than  0, 
Thus  at  completion  of  the  N  stages,  all  derivative  vectors  must  be 
lexicographically  greater  than  or  equal  to  zero. 

Q.E.D. 


Theorem  4 .3 .  The  basis  B,  obtained  at  the  end  of  Step  II  in  the 
algorithm  for  any 

where  e 


t  =  t^,  remains  optimal  for  the  interval 


tvs  +  V  ’ 


0 
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is  strictly  positive. 
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Proof  of  Theorem  4  .3  . 


f 


* 


v 


Since  each  7  (t,B)  is  a  member  of  the  class  of  solutions  to  homo- 
K 

geneous,  constant  coefficient,  linear  differential  equations,  e  it 

K 

strictly  positive  by  Theorem  4.1.  By  the  definitions  given  for  €  and 

K 

t  ,  the  basis  B  satisfies  the  optimality  criteria  for 


‘  £[to'lo  +  V 


Q.E.D. 


Theorem  4 .4 .  The  number  of  basis  changes  in  any  finite  interval  [T^,T  ] 
is  a  finite  number,  and  the  parametric  programming  problem  is 
solvable  in  a  finite  number  of  steps. 


Proof  of  Theorem  4 .4  . 

At  any  switching  point  t^,  there  exists  a  basis  and  an 

e  >  0,  so  that  B  is  optimal  for  t  e[t„,t  +  e  )  .  There  also  exists 
p  *  P  *  *00p  * 

a  basis  B  and  an  €  >  0,  making  B  optimal  for  t  e[tQ  ~  £  ,  tQ)  . 

It  follows  then,  if  there  is  a  cluster  point  at  t  ,  there  would  be  an 

r  *  i 

infinite  increasing  sequence  of  switching  points  t  eLtQ  “  e  , 
which  could  be  bypassed  by  a  single  switch  at  any  such  t.  to  basis 

■jf 

B  .  This  establishes  the  existence  of  a  finite  number  of  basis  changes 

in  any  finite  interval  t  e[T  ,T  3. 

1  2 

What  remains  to  be  shown  is  that  the  algorithm,  as  presented,  solves 
the  parametric  programming  problem  in  a  finite  number  of  steps.  As  dis- 

r  *  i 

cussed  above,  let  us  assume  there  is  a  switching  point  t  elt^  -  £  ,  t^J 

and  a  switch  from  basis  B^  ^  to  B  .  To  simplify  the  discussion,  let 

us  also  assume  that  B  differs  from  B.  .  by  the  introduction  of  one 

1  l-l 

incoming  column  k,  and  that  the  value  of  the  incoming  variable  x  = 

0  k 
x^  >  0,  i.e.,  the  basic  solution  is  nondegenerate. 

The  optimal  value  of  the  objective  z,  in  the  neighborhood  of  t^, 

takes  the  form  of 


z  =  z  ( t ,  B  , )  +  7,  ( t ,  B  ) 


0 


i-1 


i-1 


if  t  <  t. 
—  1 


if  t  >  t . 
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Both  z-(t,B.  „)  and  /,  (t.B.  „ )  can  be  shown  to  be  linear  combinations 
0  l-l  k  l-l 


of  solutions  to  homogeneous,  linear  differential  equations  with  constant 


real  coefficients  and  hence  they  themselves  are  also  solutions.  In  addi¬ 
tion  for  t+,  7  (t+,B  )  <  0,  otherwise  there  would  have  been  no  switch 

i  k  i  i-I  s 

from  B.  _  to  B..  This  means  that  the  vector  of  the  0,  1  ,2  , 

i-I  l 

s  t  — 

N-l  order  derivatives  of  7,  (t,B,  „ ) ,  evaluated  at  t.,  is  lexi- 

k  i-I  l 

cographically  negative.  It  follows  then  that  the  function  z  is  discon- 

th 

tinuous  in  t  at  t.  in  at  least  one  of  its  j  order  derivatives.  On 


the  other  hand,  the  optimal  value  of  z,  in  the  interval  t  eCt^  -  e  , t^] , 


can  also  be  expressed  as 


z  =  zQ(t, B  ) 


and  is  continuous  in  all  derivatives  at  t_^,  which  is  a  contradiction. 


The  above  argument  is  now  extended  to  the  case  where  B^  ^  is  as¬ 


sumed  to  differ  from  B  by  several  incoming  columns.  The  term 

i 


x  /  (t,B  ., )  is  then  replaced  by  a  sum  of  terms,  each  of  which  is  lexi- 
k  k  i-1 

cographically  negative  at  t  =  t^,  hence  their  sum  is  lexicographically 
negative  and  the  discontinuity  at  t  =  t^  follows.  (Note  that  the  de¬ 
generacy  of  basic  solutions  in  the  simplex  algorithm  is  assumed  to  be 
handled  by  the  standard  right-hand  side  lexicographic  rules  of  the 
simplex  method.) 

Since  each  basis  change  is  accomplished  by  a  finite  (at  most  N) 
number  of  linear  programs,  the  parametric  programming  problem  is  solv¬ 
able  in  a  finite  number  of  steps. 

Q. E.D. 


Corollary  4. 1.  The  solution  x  (t)  to  (4.1)  is  a  piecewise  constant 


vector  function  of  t  with  a  finite  number  of  discontinuities. 


Proof  of  Corollary  4.  1. 


* 

Since  x  (t)  is  constant  for  t  e[t^,t^  +  ip],  this  follows  im¬ 


mediately  from  Theorem  4.4. 


Q.E.D. 
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/JK 


Proposition  4. 1 .  When  all  the  s^  for  each  7 ^ ( t )  are  real,  the  upper 
bound  on  the  number  of  switchings  in  an  infinite  time  interval  for 
an  (m  X  m)  matrix  A  is 


i(n  -  m)  N  . 


m 


Proof  of  Proposition  4.1. 


th 


For  any  7  (t)  that  is  a  member  of  the  class  of  solutions  to 


an 


N  order  homogeneous,  constant  coefficient,  linear  differential  equa¬ 


tion  and  for  real  and  distinct  s  ,  it  is  known  [1]  that  the  function 


7  (t)  has,  at  most,  N  roots  of  7^(t)  =  0. 

At  the  most,  there  are  |  ™  j  possible  bases  for  A;  for  each  of 


these  bases,  there  are  (n-m)  nonbasic  variables.  For  each  nonbasic 


variable  x^,  the  relative  cost  factor  7i(t,B)  has  at  most  N  points 


at  which  it  crosses  the  value  zero  and  thereby  creates  a  possible  basis 
switch. 

Q.E.D. 


B.  Parametric  Quadratic  Programming 


The  conditions  for  which  the  parametric  programming  problem  has  a 
solution  are  found  in  this  section.  Also,  an  algorithm  based  on  the 
complementary  pivot  theory  procedure  for  quadratic  programming  is 
constructed. 

The  quadratic  programming  problem 


find  x  (t)  to 
minimize  7(t)'x  +  xQx 
subject  to  Ax  >  b 
x  >  0  , 


t  ett^.Tg]  , 


(4.16) 
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cnn  be  formulated  in  the  complementary  pivot  theory  as, 


*  * 

find  w  (t),  z  (t),  so  that 

w  =  mz  +  q ( t ) 

w  z  =0,  Vi 
i  i 

w.  >  0,  z.  >  0,  Vi  ,  (4.11 

l  —  l  — 

where 


2Q  -A' 

7  ( t) 

M  = 

,  q(t)  = 

A  0 

-b 

- 

„  _ 

"  X  ~ 

u  ’ 

z  = 

,  and 

w  = 

_  y_ 

V  _ 

where  y  is  the  vector  of  dual  variables  to  the  quadratic  programming 
problem,  and  u, v  are  slack  vectors  of  the  necessary  conditions  for 
quadratic  programs,  as  was  stated  in  Chapter  III.  The  necessary  con¬ 
ditions  in  (4.17)  for  Q  positive  semiriefinite  are  sufficient  at 
any  t  =  tQ.  From  the  results  of  Dantzig  and  Cottle  [3],  the  complemen 
tary  pivot  theory  algorithm  terminates  in  a  solution  to  (4.17)  when  M 
is  positive  semidef inite,  providing  the  solution  set  for 


w  =  Mz  +  q,  w  >  0,  z  >  0, 


is  nonempty. 

To  show  that  the  above  solution  set  is  nonempty,  a  solution  must 
be  shown  to  exist  for  every  t  in  the  parametric  programming  problem. 
If  such  a  solution  does  exist,  it  must  satisfy  the  conditions  of  (4.17) 
Therefore,  let  us  assume  the  absence  of  unbounded  solutions,  i.e.,  that 
the  set 
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•1 


1 


is  bounded  and  nonempty. 


Proposition  4.2.  The  parametric  quadratic  programming  problem  has  a 


solution  for  every  point  t  e[T  ,  T  ],  when  X  is  nonempty  and 

J_ 

bounded,  7(t)  is  a  vector  function  with  each  of  its  components  bounded 


in  the  interval  [T^T^],  and  Q  is  positive  semidef inite. 


Proof  of  Proposition  4.2. 


Since  the  objective  is  continuous  •>  x  over  a  compact  set  X,  it 


attains  its  infinum  at  a  point  in  X. 


Q.E.D. 


Proposition  4.3.  Given  the  above  conditions  on  /(t),  X,  and  Q,  the 


form  w  =  Mz  +  q,  w,z  >  0,  w^z .  =  0  has  a  solution  for  every 


t  e[T  , T  ],  and  this  solution  can  be  found  by  using  the  methods  of 
-L  £ 


complementary  pivot  theory  in  a  finite  number  of  pivot  operations  on  M. 


Proof  of  Proposition  4.  3. 

As  stated  above,  Dantzig  and  Cottle  [3]  have  shown  that  the  comple¬ 


mentary  pivot  theory  algorithm  converges  to  a  solution  of  the  quadratic 
programming  problem,  if  the  solution  set  for  w  =  Mz  +  q,  w,  z  >  0,  is  non¬ 
empty.  Since  X  is  nonempty,  and  since  the  infinum  of  /(t)'x  +  x'Qx 


is  attained  in  X  for  each  t,  then  the  necessary  and  sufficient  con¬ 


ditions,  i.e.,  a  solution  to  (4.17),  must  exist.  Thus,  the  conditions 


for  termination  of  the  algorithm  are  satisfied. 


Q.E.D. 


Remembering  the  characteristics  of  /(t)  explained  earlier,  we 


will  now  show  that  only  a  finite  number  of  solutions  are  considered  when 
solving  the  parametric  programming  problem  for  all  t  in  the  finite 


interval,  [T  ,T2J. 
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Theorem  4.5.  There  are  a  finite  number  of  changes  in  the  set  of  basic 
variables  of  (4.17)  for  the  parametric  programming  problem  over  a 
finite  interval. 

Proof  of  Theorem  4.5. 

Let  the  M  matrix  be  (n  X  n).  Since  each  complementary  solution 
has  n  basic  variables,  there  are,  at  most,  ( ^ j  complementary  solu¬ 
tions.  By  using  the  pivoting  procedures,  the  characteristic  of  any 
solution  is 


-  — 

w 

■ 

z 

= 

q(t) 

+ 

M 

z 

w 

wheie  w  is  the  set  of  basic  variables  of  the  vector  w, 

w 

w  = 

w 

and  z,  the  set  of  basic  variables  of  the  vector  z.  The  solution,  for 
any  t,  is 


-  _ ~ 

"  - 

- 

w 

Z 

- 

q(T> 

>  0  , 

z 

_  _ 

w 

The  vector  function  q(t)  is  of  the  form  7(t),  and  each  component  of 

q(t)  has  a  finite  number  of  zero  crossings  in  any  finite  interval  of  t. 

1  ^  i  t  h 

Let  these  points  be  labelled,  t_. .  for  the  i  complementary 

solution.  The  set  of  points 
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are  the  only  points  at  which  the  set  of  basic  variables  can  change  in 
the  finite  interval.  The  set  T  is  countable  and  has  measure  zero.  If 


the  points  are  ordered,  i.e.,  t^.t^, 


■V 


ables  that  remain  basic  must  exist  for  any  interval  [tj,t 


then  a  set  of  basic  vari- 

Th i s  is 

true  because  a  solution  has  been  shown  to  exist  for  every  t  e[t  ,t  ), 

J  J  +  1- 

thus  the  set  of  basic  variables  cannot  change  in  this  interval. 

Q.  E.  D. 


At  this  point,  an  algorithm  is  presented  to  provide  a  basic  solution 
to  the  complementary  problem;  this  solution  remains  optimal  over  a  posi¬ 
tive  interval.  The  method  used  does  not  require  prior  knowledge  of  the 
switching  points  and  does  not  assume  nondegenerate  solutions  at  these 
points.  The  algorithm  employs  the  same  pivoting  procedures  of  the  com¬ 
plementary  pivot  theory  algorithm  presented  in  Chapter  III. 

Definition  4.3.  We  will  say  a  vector  y,  which  is  lexicographically 
smaller  than  zero  (i.e.,  -y  ^  0)  lexico-increases  to  y,  if 
(y  -  y)  >-  0,  and  if  the  component  of  y  corresponding  to  the  first 


where  q^(t^)  is  the  current  value  of  the  i  component  of  qd^) 
under  the  pivoting  procedures  (i.e.,  those  procedures  leading  to 
a  particular  complementary  solution) .  Let 


e  =  inf  (  €  |  q  C  t  +  e)  <  0} 

1  f  1  u 


and  let  =  min  e  . 

0  i  1 


We  will  now  present  an  algorithm  for  solving  the  parametric  quad¬ 
ratic  programming  problem.  It  should  be  noted  that  this  algorithm  is 
also  applicable  to  the  parametric  linear  programming  problems  discussed 
in  Section  IV. A;  however,  this  method  is  more  complicated.  A  flow  chart 
and  proof  of  the  algorithm's  finite  termination  will  follow  the  general 
iteration  given  here  for  any  t  =  t^. 


Step  I:  (a)  Solve  the  complementary  pivot  theory  problem 


w  =  Mz  +  q(tQ) 
w. z .  =  0,  w,  z  >  0  , 


for  positive  semidefinite  matrix  M,  and  (b)  examine  the  nonnegative 
complementary  solution  for  t  >  t^;  i.e.,  is 


(V  *  0 

1 


or  =  0  ? 


If  Dq^Ctg)  0  or  equal  to  zero,  proceed  to  Step  III;  if  it  is  not, 
go  to  Step  II. 


Step  II:  Choose  an  index  i  so  that  Dq^Ct^)  is  a  minimum  over  all 
the  derivative  vectors  that  are  lexicographically  less  than  zero.  Retain 
all  other  variables  of  those  variables  which  have  derivative  vectors  lex¬ 
icographically  less  than  zero,  at  their  present  lower  bounds,  or  let  them 
increase  (when  forced  to  decrease  they  are  blocking  variables)  .  Then  let 
the  basic  variable  corresponding  to  i  be  the  distinguished  variable  and 
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perform  the  standard  complementary  pivot  algorithm  with  the  new  lower 
bound  restrictions;  terminate  when  the  distinguished  variable  lexico¬ 
graphically  increases  (i.e.,  when  it  drops  out  of  the  basis  by  becoming 
level  0).  The  solution  of  Step  I  is  used  to  initiate  Step  II.  Now, 
return  to  (b)  of  Step  I  with  the  current  solution. 


Step  III:  Using  the  previous  definitions,  calci 1 -te  e.  for  all  i 
and  eQ,  the  minimum  of  e^.  The  final  basis  at  tQ  is  then  optimal 
for  t  £tt0>tg  +  •  If  a  solution  for  t  >  tQ  +  is  desired,  re 

turn  to  (b)  of  Step  I,  using  the  solution  at  t  +  e^,  and  proceed  with 
the  algorithm. 

The  procedure  given  here  provides  a  solution  to  the  parametric 
quadratic  program  for  any  interval  of  the  parameter. 

It  remains  to  be  shown  that  each  step  in  the  algorithm  does  termi¬ 
nate  in  a  finite  number  of  executions  and  that  the  final  basis  generated 
at  any  t  is  optimal  over  a  finite  positive  interval  for  the  quadratic 
program.  Hence  the  remainder  of  this  section  is  devoted  to  this  proof. 

Step  I  is  solvable  in  a  finite  number  of  steps,  if  the  problem 


w  =  Mz  +  q(tQ) 
w '  z  =  0 ,  w ,  z  >  0 


can  be  solved.  Since  it  is  known  that  the  quadratic  program  has  a 
solution  for  every  t,  we  are  assured  of  the  complementary  pivot  theory 
algorithm  converging  to  an  optimal  solution  in  a  finite  number  of  steps. 


Proposition  4.4.  Step  II  must  terminate  with  a  complementary  solution 
having  the  derivative  vector  of  the  distinguished  variable  lexi¬ 
cographically  increased  while  other  variables  at  their  lower  bound 
are  not  lexico-decreased. 


Proof  of  Proposition  4.  4. 

The  complementary  pivot  theory  algorithm,  when  initiated  with  a 
basic  (complementary)  solution,  terminates  in  another  complementary 
solution,  since  we  assume  feasibility  of  the  quadratic  program.  Because 
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the  distinguished  variable  and  its  complement  are  converging  to  a  non¬ 
negative  complementary  solution,  this  termination  occurs  when  the  dis¬ 
tinguished  variable  leaves  the  basis.  When  it  leaves  the  basis,  the 
variable  has  a  derivative  vector  equal  to  zero;  thus  it  has  lexico¬ 
graphically  increased.  All  other  variables,  at  their  lower  bounds,  were 
not  permitted  to  lexico-decrease;  all  variables  entering  the  basis  are 
permitted  to  increase  only.  Thus  the  new  solution  has  no  variables 
lexicographically  less  than  zero,  and  the  distinguished  variable  is 
1 ex i co-increased. 

Q.  E.  D. 

Theorem  4.6.  The  algorithm  terminates  in  a  finite  number  of  steps  to  a 
complementary  basis  that  is  optimal  over  a  finite  positive  interval. 

Proof  of  Theorem  4.6. 

The  execution  of  Step  II  lexico-increase  at  least  one  of  the  vari¬ 
ables  that  was  lexicographically  less  than  zero  while  not  lexico- 
decreasing  any  of  them;  and  it  does  not  introduce  any  new  variables 
lexicographically  less  than  zero.  Since  there  are  only  a  finite  number 
of  lexicographically  less  than  zero  vectors  (at  most,  n)  and  since 

each  has,  at  most,  n  components,  the  execution  of  Step  II  must  termi- 

2 

nate  after  at  most  n  times  with  each  execution  requiring  a  finite 
number  of  steps.  The  termination  condition  is  that  all  basic  variables 
are  lexicographically  greater  than  or  equal  to  zero  (all  nonbasic  vari¬ 
ables  lexicographically  equal  to  zero).  Thus  by  Theorem  4,1,  the  basic 
variables,  and  hence  the  solution  to  (4,17)  are  nonnegative  over  a 
finite  positive  interval. 

Q.  E.  D. 

From  the  above  results,  it  has  been  determined  that  a  finite  number 
of  basis  changes  are  required  to  solve  the  parametric  quadratic  program¬ 
ming  problem  for  a  finite  interval  of  the  parameter.  The  solution  to 
the  parametric  quadratic  program  need  not  be  piecewise  constant  as  it  is 
in  the  parametric  linear  programs.  For  each  interval  where  a  single 
basis  remains  optimal,  the  solution  will,  in  fact,  have  the  character¬ 


istic  form 


where  w  and  z  re  the  complementary  variables  in  their  original  form 
and  z  and  w  are  their  complements.  q(t)  is  nonnegative  over  the 
interval  and  has  the  form  where  each  of  its  components  solves  some  par¬ 
ticular  order  homogeneous  constant  coefficient  linear  differential 
equation.  When  the  value  of  some  q^(t)  goes  negative;  for  some  t, 
the  basic  variables  w, z  are  no  longer  optimal,  and  a  new  set  of  basic 
variables  must  be  found  with  the  complementary  property.  The  new 
values  of  q^(t)  are  just  linear  combinations  of  the  former  components. 


Chapter  V 

GENERALIZED  PROGRAMMING  ALGORITHM  FOR 
OPTIMAL  CONTROL  PROBLEMS 


The  mathematical  programming  results  obtained  in  the  previous  two 
chapters  are  applied  to  the  linear  system,  continuous-time  optimal  con¬ 
trol  problems  to  formulate  [9]  a  generalized  linear  program.  A  solution 
procedure  based  on  this  formulation  is  then  developed  and  is  shown  to 
terminate  in  an  optimal  solution,  i.e.,  an  optimal  control  is  provided 
to  the  continuous-time  problem. 

A.  Formulation 


The  control  problems  will  now  be  formulated  as  generalized  programs 
and  then  the  subproblems  will  be  shown  to  be  parametric  programming  prob¬ 
lems  of  the  form  presented  in  Chapter  IV.  The  control  problem  can  be 
restated  as 


min  J  =  f  x  (t)  dt  =  x  (T) 

u  (  • )  Jo  U  U 


(5.1a) 


x(t)  =  Fx(t)  +  Gu(t) 


(5.1b) 


x  e 


„m 

u  e  E  , 


and 


x 


e 


En+1 


x(0)  e  SQ  ,  and  x(T)  e  ST  , 

u(t)  e  U  =  (u|Au  >  b]  ,  U  c  Em  , 

F  is  an  n  X  n  real  matrix,  and 
G  is  an  n  X  m  real  matrix. 
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Using  the  above  set  of  definitions,  we  can  restate  the  initial  and 
final  state  constraints  of  the  fixed  time  control  problems  as 


x(0)  =  0  ,  x  (0)  =  0  ,  and 


x(T)  e  &  , 


Thus,  it  can  be  assumed,  without  loss  of  generality,  that  the  system 
initiates  at  the  origin  with  no  prior  costs. 

1 .  Control  Problems  Formulated  as  Generalized  Programs 

If  we  take  the  vector  functionals  of  the  control  P  =  P[u(t)] 
to  be  defined  by 


P=  f  e<T-t)F  Gu(t)  cl.  , 


(5. 5a) 


P  =  P[u(t)]  to  be  defined  by 


P=  f  e(T-t)PSu<t)  cl.  , 


(5.5b) 


then,  let 


s  '  P  u(t)  e  U  , 


P  =  f  e(T  t)F  Gu( t)  dt 


C.|p  u(t)  e 


P  =  f  Gu(  t)  dt 


Proposition  5.2.  If  U  is  a  convex  set,  the  set  C(C)  is  convex. 


Proof  of  Proposition  5.2. 

i  2 

Let  u  (t),  u  (t)  be  vector  functions  in  U,  for  all  t,  and 
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P1  =  f  e(T_t)F  GuX(t)  clt  c  C 

Jo 


P2  =  f  e(T-t)FGu2(t)  dt  e  C 

Jo 


For  all  A,  0  <  A  <  1,  A  =  (1-A), 


AP1  +  AP2  =  A  f  e(T_t)F  Gu1(t)  dt  +  A  f  e(T_t)F  Gu2(t)  dt 

Jo 


e(T-t)F  G^u1^)  +  Au2(t)  ]  dt 


1  —  2 

Since  U  is  convex,  Au  (t)  +  Au  (t)  £  U,  Vt. 
1  —  2 

Thus  AP  +  AP  e  C,  and  C  is  convex. 


Q.E.D. 


Remembering  that  the  state  at  time  0  is  assumed  to  be  at  the 
origin  and  using  the  definitions  of  P,  C,  and  Eq.  (5.4b),  we  find  C 
is  equivalent  to  Rt>  the  reachable  set  of  U  at  time  T.  The  control 
problem  is  feasible,  iff 


CDS  ^  0  . 


Given  a  specified  control  function  u(t),  the  cost  associated  with  that 
control  is  J[u(t)],  since  the  cost  at  time  t  =  0  is  zero.  Note  that 


m  „ 


■po" 

t 

p 


where  pQ  =  J[u(t) ]  , 


when  u(t)  is  the  control  generating  P  by  Eq.  (5.5a).  Thus 
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p 

Zi  *  v  n  ’J  -T  •.  -  V  V  - 


—  n+1,— 

P0 

! 

c  = 

P  £  E  |  P  = 

P 

,  P  =  P(u)  , 

P0  =  J(u)  , 

u(t)  t  U  | 

Note  also  that  by  using  Proposition  5.2,  C  is  convex. 

Let  Uq  =  (1,0 . 0)  and  note  that  the  first  component  of 

the  vectors  S,  in  the  set  A,  is  defined  to  be  zero.  Also  note  that 
the  first  component  of  the  P  vector  represents  the  cost  of  using  the 
control  (and  its  corresponding  trajectory)  generating  P.  Thus  we  are 
looking  for  a  vector  P  e  C,  a  vector  function  u(t)  generating  P, 
and  a  vector  S  e  ^  to  satisfy 


_max  A 
P  e  C 


subject  to 


i,  v  >  0 


U0A  +  PM  =  SV 
p  =  1 
V  =  1  , 


(5.6) 


where  p  and  v  are  scalars.  Maximizing  A  is  equivalent  to  minimiz¬ 
ing  J[u(t)] ,  the  first  component  of  the  vector  P,  where  u(t)  gen¬ 
erates  P.  Since  P  must  be  taken  from  a  convex  set  C  and  S  must 
be  taken  from  a  convex  set  A,  the  above  formulation  is  a  generalized 
program  of  the  Dantzig-Wolfe  type.  In  the  following  chapter  we  will 
show  that  an  optimal  solution  to  the  control  problem  is  an  optimal  solu¬ 
tion  to  the  generalized  programming  problem.  We  will  now  show  that  any 
solution  to  the  generalized  programming  problem  is  an  optimal  solution 
to  the  original  control  problem. 

A  solution  to  the  generalized  programming  problem  consists  of 
a  vector  P  in  the  reachable  set  R^,  a  control  function  u(t)  in  the 


admissible  control  region  U  generating  P,  and  a  vector 
constraint  set  of  terminal  states  A,  so  that 

P  =  S  . 


in  the 


SEL-68-085 


60 


The  above  equality  insures  the  transformation  of  the  system  from  an 
initial  point,  x(0)  e  to  a  final  point  x(T)  e  by  the  vector 

function  u(t),  chosen  from  U.  Thus  it  is  a  feasible  control.  By 
minimizing  J  over  all  feasible  sets  of  P  and  S,  we  can  find  a 
feasible  solution  with  the  least  cost.  This  is  precisely  an  optimal 
solution  to  the  continuous-time  control  problem. 

2.  Generalized  Programming  Subproblems  as  Parametric  Programs 

To  complete  the  generalized  programming  formulation,  its  sub¬ 
problem  must  be  described.  Here  we  assume  there  are  at  least  n  +  2 
vectors  P1  and/or  S1  available  to  provide  a  feasible  solution  to 
(5.6),  so  that  the  problem 


max  A  , 
(i»  v 


p,  v  >  0 


r,  -X  ,  n1  L  r,2  ^  r,J  CJ  +  1  ^  CJ  +  P 

U0A  ^  P  M2  P  =  S  vi  +  •  •  •  +  S  vp 


J1  +  P2  +  ■  ■  •  +  Mj  =  1 


V-  +  V0  +  •  •  •  +  V  -  1  i 

12  p 


(5.7) 


is  solvable  and  has  a  dual  solution  vector 


-  <V”'’ Vl'%+21  ' 


where  rt'  =  (it„,...,Tr  ).  The  subproblem  is  then  formulated  in  two  parts: 
1  n 


min  tt'  0  ,  and 

S  e  I 


min  jt'  1 
P  e  C 


(5.8) 


(5.9) 
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The  solution  to  (5.8)  is  dependent  on  the  explicit  definition  of  the  set 
if  or  A,  the  simplest  case  being  the  fixed  end  point  problem,  which 
consists  of  a  single  element.  In  this  case,  the  subproblem  (5.8)  is 
trivial  and  need  not  be  considered.  If  A  is  a  convex  polyhedral  set, 
then  (5.8)  is  a  linear  program  that  needs  to  be  solved  once  for  each 
iteration  of  t  lie  master  problem: 


Subproblem  (5.9)  can  be  described 


min  it'  1  =  min  it' 

P  t.  C  P  e  C 


(X-t)  F  - 
e  Gu( t )  dt 


(5.  10) 


Since  the  requirement  P  t  C  is  equivalent  to  the  requirement  u(t)  c  U 
for  all  t,  (5.10)  becomes 


(X-t) F  — 
e  Gu(t)  dt 


min  it' 


u ( t)  t  U 


since  rt  does  not  depend  on  t, 


u  ( t )  t  U  I  -/0 


(it  ,7t')  e^T  ^  F  Gu(  t)  dl  +  it 
0  n+1 


(5. 11) 


The  minimum  of  the  integral  is  attained  when  the  integrand  is  minimized 
at  every  point.  Let 


(T-t)F  - 

/  ( t)  =  (Tt0,TT')  e  G 


(5.12) 
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be  an  m-dimensional  vector  function.  Thus  the  subproblem  becomes 


Find  u(t)  e  U  ,  t  e[0,T] 

so  that  /(t)'u(t)  is  a  minimum  ,  (5.13) 

From  (5.12),  it  is  obvious  that  7(t)  has  the  property  of  each  of  its 

st 

components  being  a  member  of  the  class  of  solutions  to  an  n  +  1  order, 
homogenous,  constant  coefficient,  linear  differential  equation.  Since 
our  attention  is  restricted  to  those  U  that  are  polyhedral  sets,  (5.13) 
becomes 

min  7 ( t) ' u( t) 

Au ( t)  >  b 

t  e [0, T]  .  (5.14) 

(Note  that  the  inequality  may  be  reversed  or  an  equality  without  loss  of 
generality.)  Thus  a  solution  u(t),  for  the  subproblem,  can  be  obtained 
by  using  the  parametric  linear  programming  methods  of  Chapter  IV. 

In  a  similar  manner,  we  can  formulate  minimum  fuel,  minimum 
time,  and  quadratic  loss  in  control  problems  as  generalized  programs.  It 
can  also  be  shown  that  the  minimum  fuel  and  minimum  time  are  special 
cases  of  the  linear  loss  problems  just  described.  Since  generalized  pro¬ 
gramming  can  be  applied  to  general  convex  programming  problems,  we  can 
formulate  optimal  control  problems  with  loss  functions  convex  in  the  con¬ 
trol  variable  as  generalized  programs.  However,  only  the  quadratic  loss 
in  the  control  case  will  be  discussed  in  detail,  since  this  is  (to  the 
author's  knowledge)  the  only  general  nonlinear  convex  loss  function  which 
has  a  known  finite  solution  procedure  for  the  parametric  subproblem.  Sep¬ 
arable  piecewise  linear  (convex)  functions  of  the  control  can  also  be 
formulated  as  a  special  case  of  the  linear  loss  problem,  although  it  will 
not  be  shown  here. 


The  previous  formulations  can  be  generalized  if  we  observe  the 


following  general  linear  system  control  problem  and  use  the  notation 
given  in  Chapter  II, 


min  J 
u  ( t )  &  U 


T 

-  *c 

Jo 


(t)  dt, 


where  x(t)  = 


E„+i 

t  E 


x(t)  =  Fx( t)  +  Gu(t)  +  f(u)  UQ  ,  (5.15) 


,  linear  loss 


where  f(u)  =  (  £  |  u  I  ,  minimum  fuel 
1  1 

I  u'Qu  ,  quadratic  loss  , 


where  Q  is  positive  semidef ini te,  u(t)  e  U,  t  t[0,T],  x(0)  e  SQ, 
x (T)  ,  and  SQ,  are  convex  sets  in  En.  We  note  that  J  is 

a  convex  functional  in  u(t),  since  u(t)  is  a  vector  sequence  drawn 
from  a  convex  set  U,  and  f(u)  is  a  convex  function  in  u.  Thus  the 
solution  to  Eq.  (5. 15)  can  be  noted  as 


x(T)  =  eFT  x ( 0)  +  f  eF(T"°  Gu(t)  dt  +  f  e'  _t  f[u(t)]  U  dt  . 

Jo  Jo 

(5.16) 


Now,  we  define  P  and  P,  as  before,  by 


f  F(T-t) 

P  =  |  e  Gu(t)  dt  ,  and 


(5.17) 


p  T  —  T 

=  f  eF(1"1)  Gu(t)  dt  +  f  f[u(t)]  dt  U  ,  (5.18) 

p  Jo 
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is  an  n  +  1-dimensional  unit  vector  with  a  one  in  the  first 


where  U 


0 


component.  Thus  the  second  integral  of  (5.18)  becomes 


m 


L  ^ 


(t)ldt  ,  minimum  fuel  , 


i=l 


•'0 


(t)'Qu(t)  dt  ,  quadratic  loss 


Also  as  was  done  before,  we  can  now  define  C  as 


C  =  /  P  e  E 


n+1 


P  = 


/ 

J0 


gF(T-t)  Gu(t)  dt 


z  >  J[u(t)]  ,  u(t)  e  U,  V t j 

Thus  the  P,  as  defined  in  (5.18),  are  members  of  C.  We  also  note 
that  C  is  a  convex  set,  since  J[u(t)]  is  a  convex  functional. 

The  vectors  S  and  set  A  are  defined  as  before.  Thus  the 
general  linear  system  control  problem  can  be  formulated  as  a  generalized 
programming  problem, 

Find  P  e  C  ,  S  e  A  ,  to 


max  A  ,  p,  v  >  0 
P  e  C 


subject  to  UqA  +  Pp  =  Sv 


p  =  1 

v  =  1 


(5. 19) 


Again  we  note  that  the  solution  to  the  generalized  program  is 
_*  * 
one  in  which  a  vector  P  is  found  so  that  the  last  n  elements  P 

belong  to  the  set  A,  and,  out  of  all  possible  vectors,  P  e  A,  i.e., 
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feasible  solutions  to  the  control  problem;  the  first  component  of  P  , 
taken  to  be  the  value  of  the  loss  functional,  is  minimal.  Thus  the 

■X-  — -X- 

vector  control  function  u  (t)  generating  P  is  a  solution  to  the 
optimal  control  problem. 

We  now  describe  the  subproblem  corresponding  to  the  minimum 
fuel  and  quadratic  loss  problems  as  in  Eqs.  (5.8)  and  (5.9).  For  the 
general  case,  following  the  similar  reasoning  given  for  the  linear  loss 
case,  the  subproblem  to  the  restricted  master  problem  becomes,  (remember¬ 
ing  that  n  =  1,  since  (Uq,0,0)  is  a  basis  vector  and 


it*  0  1=1) 


lin  f  jfru(t)]  +  (it  it')1  e 
i(-) 


’)'  e(T  t)F  Gu ( t )  >  dt  +  it  ,  .  (5.20) 

1  n+1 


Defining  /(t)  as  before,  (5.20)  becomes 


min  /(t)'u(t)  +  f[u(t)] 
u  ( t ) 


subject  to  u(t)  t  U  ,  Vt  . 


(5.21) 


For  the  quadratic  loss  problem,  with  U  =  [ujAu  >  b,  u  >  0}  , 
(5.21)  becomes  a  parametric  quadratic  programming  problem  in  u(t)  of 
the  form  discussed  in  Chapter  IV  [due  to  the  form  of  y(t)]. 

We  now  look  at  the  minimum  fuel  problem  for  two  classes  of  U. 
The  first  class  has  the  classical  form  of  the  minimal  fuel  problem, 
where 


U  =  {  u 1 1  u  ( t ) [  1,  i  =  1,  ....  m]  , 


and  the  second  class  is  a  general  polyhedral  U. 
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For  the  first  case,  (5.21)  has  the  following  solution  (the 
singular  arcs,  y(t)  =  1  for  an  interval,  are  not  discussed  because  no 
solution  is  defined) , 


7  .  (t) 
i 

<  1  , 

u .  (t) 

l 

=  0 

/-N 

+-> 

•H 

<  -1  , 

u.  (t) 
i 

=  1 

7.(t) 

>  1  , 

U .  (t) 

1 

=  -1 

for 

all  i  . 

(5.22) 

The  magnitude  of  one  as  a  bound  for  the  control  is  noted  to 
be  nonrestrictive,  since  G  can  be  scaled  to  permit  other  values.  The 
formulation  can  also  be  adapted  to  treat  lower  bounds  on  u.(t)  with 
magnitudes  that  differ  from  the  upper  bounds.  These  changes  affect  the 
ranges  of  7_(t)  in  (5.22).  Thus  the  subproblem  for  the  standard  mini¬ 
mum  fuel  problem  has  a  well-defined  solution,  and  its  execution  in  re¬ 
lation  to  the  master  problem  is  proportionately  as  quick,  regardless  of 
the  size  of  the  control  space. 

The  minimum  fuel  problem  for  general  polyhedral  sets  U  has 
a  subproblem  equivalent  to  that  of  the  linear  loss  case.  Following  the 
same  steps  given  above,  the  subproblem  for  general  U  which  replaces 
(5.23)  is 


min 

u(-) 


7  ( t) u( t) 


subject  to 


Au(t)  >  b 


0  <  t  <  T 


(5. 23) 


This  is  equivalent  to  a  parametric  linear  programming  problem  of  the 
type  presented  in  Chapter  IV  when  the  variable  u  (t)  ,  which  is  un¬ 
restricted  in  sign,  is  replaced  by  the  difference  between  two  nonnega¬ 
tive  variables, 
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The  constraints  are  replaced  by 


Au  -  Au  >  b 

u  >  0 

u  >  0  .  (5.25) 

Since  linear  programming  algorithms  consider  only  basic  solutions, 

and  u.  cannot  be  basic  at  the  same  time  because  their  columns  A.  and 

l  i 

-A.,  respectively,  are  linearly  dependent.  Thus  for  every  i,  either 

u.  or  u.  must  be  at  level  zero.  Using  this  result,  |u.(t)|  can  be 
ii  1  l  1 

replaced  by 


u.(t)  =u.(t)+u.(t), 

l  ii 


(5. 26) 


and  the  equivalent  parametric  linear  program  is 


min 
u,  u 


C/  i  ( t )  +  1]  ui(t)  +  ^  [-7  (t)  +  1]  u^t) 

i  i 


subject  to  Au(t)  -  Au(t)  >  b 

u.(t),  u.(t)  >  0 
i  l  — 

for  0  <  t  <  T  .  (5.  21) 


Since  a  generalized  programming  formulation  is  shown  to  be  used 
for  finding  a  feasible  solution  to  the  linear  system  control  problems, 
the  minimal  time  Droblem  can  be  solved  with  these  methods.  The  subprob- 
letn  to  the  generalized  programming  problem  for  feasible  solutions  is  also 
shown  to  be  a  parametric  linear  programming  problem  of  the  type  discussed 
Here  we  will  present  a  solution  procedure  (similar  to  the  one  proposed 


by  Ho  [11])  to  the  minimal  time  problem  that  is  equivalent  to  finding, 
if  it  exists,  a  feasible  solution  to  the  generalized  programming  form- 
^  ulation  of  linear  systems  and,  of  course,  to  determining  whether  or  not 

a  feasible  solution  exists. 

The  minimal  time  problem  can  be  stated  as 

min  |  f  dt  =  t 

u ( • )  Jo  f 

(t  is  free), 

where  x(0)  is  given  and  x(tf)  =  0,  and 
x(t)  =  Fx ( t)  +  Gu(t) 

x(t)  €  En,  u( t)  e  U  c  E1"  .  (5.28) 

Thus  we  wish  to  find  the  shortest  time  in  which  we  can  transfer  the  state 

•  from  a  given  point  to  the  origin  or,  equivalently,  to  find  the  minimum 

time  for  which  a  control  u(t)  e  U  exists  to  transfer  the  system  from  a 

given  point  to  the  origin.  The  latter  statement  is  the  one  related  to 

a  generalized  programming  formulation  of  the  minimal  time  problem.  De- 

TF 

fine  P  as  before,  for  any  fixed  T,  and  S  =  -e  x(0)  .  Let  the 
set  of  all  P  be  C  =  R^,  i.e.,  the  reachable  set,  which  is  convex. 

We  want  to  know  whether 

S  e  R^  ,  for  any  fixed  T  ,  (5.29) 

so  that  we  can  find  the  minimum  T  for  which  (5.  29)  holds.  Equiva¬ 
lently,  we  want  to  know  whether  a  solution  exists  for 

Pp  =  S 
M  =  1 

P  e  C  =  ,  for  any  T  ,  (5.30) 
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and  again  we  want  to  find  the  minimum  T  for  which  a  solution  to  (5,30) 
exists.  The  solution  procedure  for  (5.30)  for  any  T,  will  be  discussed 
later.  At  this  point,  we  will  solve  the  minimum  time  problem  by  choosing 
a  T,  and  try  to  solve  (5.30).  If  a  solution  exists,  decrease  T  and 
proceed;  if  one  does  not  exist  then  increase  T  and  proceed.  If  the 
increments  for  the  increase  and  decrease  of  T  are  cnosen  wisely,  this 
procedure  will  converge  to  an  answer  to  the  minimum  time  problem. 

The  solution  to  (5,30)  is  a  phase  I  generalized  programming 
procedure  that  is  also  used  to  find  initial  feasible  solutions  to  the 
optimal  control  problems  discussed  here.  A  finite  convergence  procedure 
is  shown  for  phase  I  methods  when  its  existence  is  known,  and  a  test  for 
its  existence  will  be  presented  for  control  problems  for  which  the  exis¬ 
tence  of  feasible  solutions  is  not  assumed. 

To  generalize  and  summarize  the  above  results,  the  following  class 
of  control  problems  may  be  formulated  as  generalized  programming  problems: 


min  J 
u(-) 


-f 

Jn 


f (x , u)  dt  , 


x  =  Fx  +  Gu, 


x(0)  £  S 


x(T)  £  S„ 


u(t)  £  U 


0  <  t  <  T 


(5.31) 


where  S 


0’ 


J  , 

T 


and  U  are  convex,  and 


f  ( X ,  u)  =  f1(x)  +  f2(u) 


where 
in  u 


fx(x) 


is  linear  in  x  and  independent  of  u, 
and  independent  of  x.  When  S^,  S  ,  and  U 


f2(u) 


is  convex 
are  polyhedral 


sets  and  f  (u)  cluac^rat: u  or  the  sum  of  the  absolute  value  of 

the  components  (with  linear  terms  permitted),  the  generalized  program¬ 
ming  problem  is  solvable  by  the  methods  presented  in  the  previous  two 
chapters.  The  rest  of  this  chapter  is  devoted  to  the  development  of  the 
algorithm  for  solving  these  generalized  programming  problems  and  to 
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pointing  out  the  specific  features  of  the  algorithm  so  that  it  can  be 
adopted  for  special  purposes,  including  the  determination  of  feasibility 
or  its  existence. 


f'  < 


vo- 


B.  Solution  of  the  Control  Problem 

The  first  step  and,  at  times,  the  major  problem  in  the  solution  of 
the  control  problem  is  to  find  an  admissible  control  yielding  a  feasible 
solution.  An  important  characteristic  of  the  generalized  programming 
solution  of  optimal  control  problems  is  that,  at  every  stage  in  the 
optimization  phase,  a  feasible  solution  is  always  available.  With  this 
feasible  solution,  a  bound  on  the  value  of  the  optimal  objective  func¬ 
tion  can  then  be  computed.  Thus,  if  the  solution  procedure  is  inter¬ 
rupted  before  its  convergence  to  an  optimal  solution,  a  feasible  solution 
can  be  recovered  and  an  estimate  of  how  close  it  is  to  an  optimal  solu¬ 
tion  provided.  This  estimate  or  bound  can  be  used  to  terminate  the  al¬ 
gorithm,  since  suboptimal  solutions  having  an  objective  value  close  to 
the  optimal  one,  are  generally  sufficient  for  decision  purposes.  Al¬ 
though  the  general  solution  to  the  linear  control  problems  may  be  an 
infinite  convergent  process,  the  generation  of  a  feasible  solution,  if 
interior  solutions  exist,  is  a  finite  process,  and  the  generation  of  a 
suboptimal  solution,  as  close  as  desired  in  objective  value  to  the  op¬ 
timal,  can  also  be  obtained  in  a  finite  number  of  steps.  The  algorithm 
and  its  variants  are  presented  in  this  section  along  with  convergence 
and  finiteness  proofs.  The  characteristics  of  solutions  and  their  re¬ 
lations  to  known  results  in  control  theory  are  presented  in  Chapter  VI. 

1 .  Generation  of  a  Feasible  Solution 

There  are  two  major  aspects  of  finding  a  feasible  solution. 

The  first  of  these  is  the  determination  of  whether  or  not  a  feasible 
solution  exists;  the  second  is  to  generate  the  feasible  solution,  if  it 
does  exist.  In  both  cases,  a  phase  I  procedure  of  the  generalized  pro¬ 
gramming  problem  is  uSed.  The  control  problem  to  be  considered  is 


Find  a  u(t)  €  U  =  {u  g  E  |  Au  <  b)  , 


Vt  , 
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and 


such  that  x(0)  e 

x(T)  e  S  ,  when  x  is  controlled  by 

x  ( t )  =  Fx( t)  +  Gu(t)  .  (5. 32) 

Without  loss  of  generality,  we  let  -  0  and  =  S . 

Let  us  assume  that  the  reachable  set  is  a  continuum.  Let 

us  also  assume  that,  if 

rt  n  S  *  0  , 

then  R  D  J  is  a  continuum.  This  condition  insures  the  existence  of 
a  finite-dimensional  neighborhood  in  that  set  of  desired  final  states 
in  the  reachable  set.  As  will  be  shown,  these  conditions  imply  that  the 
phase  I  portion  of  the  generalized  programming  formulation  of  the  control 
problem  terminates  with  a  feasible  solution  in  a  finite  number  of 
of  steps. 

Since  the  convergence  (Chapter  III)  of  the  generalized  program 
assumes  that  a  nondegenerate  feasible  starting  solution  is  available, 
the  phase  I  procedure  must  terminate  with  such  a  nondegenerate  feasible 
solution.  This  implies,  for  an  n-dimensional  state  space,  that  a  col¬ 
lection  of  n  +  1  vectors  P  must  be  generated  so  that  the  initial 
basis, 


is  nonsingular.  Also  a  value,  S  t  & ,  must  be  provided  so  that  B  S 
is  a  vector  and  is  strictly  positive  in  all  components. 

Although  convergence  proofs  require  the  results  of  the  follow¬ 
ing  phase  I  procedure,  more  efficient  variants  (to  be  presented)  should 
be  used  in  practice  and  can  provide  a  feasible  solution  in  fewer  steps 
for  most  problems. 


VW*«4. 


We  will  now  show  the  procedure  for  finding  a  feasible  solution 
when  S  is  a  single  point  and  a  ball  of  radius  p  in  En  (a  p- 
neighborhood)  is  also  contained  in  the  reachable  set,  R^.  The  p- 
neighborhood  is  used  to  avoid  degeneracy  problems,  in  much  the  same  man¬ 
ner  as  a  simplex  lexicographic  methods  by  perturbating  the  original  right- 
hand  side.  Thus  we  seek  to  generate  a  set  of  n  +  1  vectors  p1,  to 
provide  a  nondegenerate  solution  to  the  set  of  linear  equations. 


—  p"+1w  ■ s 


^1  +  • • •  +  ^n+1 


p,  >  0  ,  Vi 


(5. 33) 


a.  Some  Properties  of  Convex  Sets 

Definition  5. 1.  The  convex  hull  of  a  set,  X  q  En,  is  the  intersection 
of  all  convex  sets  in  En  containing  X. 


Definition  5.2.  The  convex  hull  A  of  a  finite  set  of  n  +  1  points, 


xl'  X2' 


x  ,,  ,  in  E  is  an  n-dimensional  simplex,  if  the 
n+1 


flat  of  minimal  dimension  containing  A  has  dimension  n.  The 
points  x^  are  called  vertices. 


Lemma  5.  1  [12] .  If  A  is  an  n-dimensional  simplex  with  vertices 

x  (i  =  1 . n+1),  then  A  consists  of  all  points  x  e  E1 

i 

for  which  constants  CC,  exist,  so  that 

i 


*  -  2  Vt  ■  1ai  = 


a  >  o  . 

i  — 
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Definition  5. 3.  A  set  of  k  +  1  points  in  E  is  geometrically  in¬ 
dependent,  if  no  (k-1) -dimensional  hyperplane  contains  all  the 
points . 


Definition  5 .  4.  A  set  [x^,  x^)  of  vectors  in  E  is  point- 

wise  independent  (algebraic  counterpart  of  geometrically  indepen¬ 
dent),  if  the  k  vectors,  x  -  x  ,  x  -  x  ,  .  ..,  x  -  x  are 

1  u  Ci  0  k  0 

linearly  independent. 


Lemma  5 . 2  [13].  If  X  =  (x^,  x^,  ...,  x^]  is  a  pointwise  independent 


set  in  E  ,  then  there  exists  a  unique  k-dimensional  hyperplane 


k  k 

H  containing  X  having  the  property  that  x  e  H  ,  iff 


=  \  ax. 

Z,  1  1 

i=0 


V 


a  =  l 

i 


i=0 


where  the  a,  are  unique.  The  a.  are  the  barycentric  coordi¬ 


nates  of  x  with  respect  to  X. 


Let  us  look  at  the  convex  hull  of  the  set  of  n  +  1 


points  in  E  ,  X, 


x  =  (x0>  k1> 


Xn}  ’ 


where 


X0  = 


r  '"i 

0 

1 

0 

0 

0 

0 

1 

0 

• 

II 

• 

X 

to 

II 

• 

,  ...,  xn  = 

• 

0 

0 

0 

1 

—  — 

_  _ 

_  _ 

and  denote  it  by  convX.  These  points  are  pointwise  independent,  thus 
their  convex  hull  is  a  simplex 


1 


* 

X  = 


thus  x  e  convX,  since  the  barycentric  coordinates  are  OC  =  1/n+l,  Vi. 

*  * 

Also  since  0!^  >  0,  Vi,  x  is  in  the  interior  of  convX. 

Define  a  new  set  of  points, 


X*  =  fx0'  xl’ 


...  g  , 


where  x'  =  x.  +  Ax  , 
i  i  i 


is  as  before,  and 


l^ill  <  - "2  • 

(n+1) 


Lemma  5.3.  The  points  x^  are  pointwise  independent. 


Proof  of  Lemma  5.3. 

Let  x"  =  x,  +  Ax  -  Ax  ,  j  =1,  ...,  n.  Assume  that  the  x"  are 
3  J  J  ^  J 

linearly  dependent.  Then  a  nontrivial  set  of  A  exists,  so  that 

J 


Thus, 


'V  A  x'.'  =  0 
Z,  J  J 


\  A  .x"  =  \  A.[x.  +  Ax.  -  Ax  ]  =  0  , 

L  J  J  Zs  J  3  J  o 
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which  implies  1  <  2n/(n+l)  ,  which  is  a  contradiction  for  n  >  0. 

Thus  the  vectors  x'  are  pointwise  independent  or  geometrically  in- 
dependent. 

Q.E.D. 

* 

Theorem  5.1.  The  point  x  is  in  the  interior  of  convX'  . 

Proof  of  Theorem  5.1. 

ConvX'  forms  a  simplex,  since  the  points  x^  are  pointwise  in¬ 
dependent  and  form  an  n-dimensional  hyperplane.  Thus 

n 

x  =  £  afXi  ’  and 

i=0 


I 


cc  =  1 


(5. 36) 


has  a  unique  solution  in  the  Ct^. 

We  must  now  show  that  >  0,  Vi.  Notice  that 


I 

i=0 


— -  xj  =  x  = 
n+1  i 


I 


Ci'  x' 

Vi  ’ 


or 


n  1 

y  a’x'. - i-  X  =  0 

i  i  n+1  ij 

i=0  L 


(5. 37) 


Since  x^  =  x^  +  Ax^,  (5,37)  becomes 


I 

i=0 


— 

~ 

[a1 

-  \) 

x  +  a' Ax. 

[1  1 

n+1  j 

i  i  i 

=  0 


(5. 38) 
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£  (“1  -  itt)  +  I  “;<Ah  l>  - 

i=l  '  '  1=0 


which  implies  by  using 


l-Q!' 

1  o  ’ 


y  = 

/ .  n+l  n+1  , 


1  -  %  - 1  +  si + 1  i>  -  °  ■ 


=  ^r+  S  i> 


2 

Remembering  that  ||/scj|  <  l/(n+l)  and  ||l|j  = -Jn,  we  obtain 


a*  >  — _ \ 

0  >  n+l  ,  .,2  Z 

(n+D 


2  a;  = 


1  sfn 


n+l  .  .2 

(n+l) 


Thus  CC'q  >  0,  and  x  is  in  the  interior  of  convX'  . 


Q.  E.  D. 


By  using  the  assumption  that  a  p -neighborhood  about 


S  =  X^  is  also  in  R^,  the  following  set  of  points  are  found  in  R^, 
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mm mm 

wgjygtf 


m™ 


m 


ISi 


(  — —  \  1  H 

-  e  ,  i  : 

=  0  1 

nl 

\  n+1  )  - 

n+1  i’ 

^  t  x »  *  • 

”  / 

where  e.,  1=1,  ...,  n  is  an  n-dimensional  unit  vector  with  a  one  in 
,  th  1 

the  i  row  and  e  is  the  null  vector.  Geometrically,  X  defines 
u  P 


the  vertices  of  an  n-dimensional  simplex  with  XT  defined  by  the  bary- 


centric  coordinates  of  l/(n+l),  for  each  point  in  X  . 


Let  X  =  X - —  (  — 

i  T  n+l\n+ 


(&)i 


+  Ar  e.  , 
n+1  1 


Proposition  5.6.  Let  a  ball  of  radius  [p/(n+l)  ]  [l/(n+l)  ]  ,  about  X. 


for  any  i,  be  fL,  then 


N  c  • 
i  —  T 


Proof  of  Proposition  5,6. 

xi  -  \  =  -ktt(stt)  i 


n+1  i 


x  ■  "  XTH  <  II -V  (  )  1  II  +  \\~^T  e  •  I 

l  T11  —  11  n+1  \  n+1  /  —  11  "n+1  i1 


1  /  P  \  / —  ,  P  P  ( \/"n+n+l 


[i  -  Ml  S 


n+1  n+1  \  n+1 


The  maximum  distance  from  any  point  in  N.  to  X.  is  — — i— 

J  l  i  n+1  n+1  ’ 


thus  the  maximum  distance  from  any  point  in  to  X  is  d, 


(srr)  +  ~  CPK^«+1]  , 

(n+1) 


d-P_<„+i)2^' 


+  n  +  v  n 


which  is  less  than  p  for  n  >  1.  Thus  c  since  a  p-neighbor- 


hood  about  X  is  in  R  . 

T  T 


Q.  E.  D. 
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X  e  X  : 
i  P 


We  will  now  solve  the  following  problems,  for  all 


Solution  Procedure  for  Feasible  Solution  to  a 
Generalized  Program 


Find  a  vector  P  (or  a  convex  combination  of  vectors), 


so  that 


n+l  yn+1  J 


-l  n 


+  iy  -  iy  =  x. 


M»  yi  I  yi  >  0  »  p  6  c 


(5. 39) 


This  is  the  phase  I  procedure  of  a  generalized  program.  The  columns 
p1  can  now  be  generated  to  the  master  program  (a  linear  program), 


min  w 
M-y 


=  I  + 1  yi 


plu1  +  • . .  +  pkpk  +  iy+  -  iy  =  x. 


px  +  ...  +  nk 


y±>  yl  >  0 


(5.40) 
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_j^  ^ 

From  the  vector  of  dual  variables  it  ,  the  k  iteration  of  the  general- 

k+1 

ized  program  (5.40)  generates  a  new  column  P  ,  by  means  of  the  sub¬ 
problem, 


min  — k 

_  _  Tt 

P  e  C 


(5.41) 


This  subproblem  is  equivalent  to  finding  a  vector  function  u  (t)  that 


.  .  k’  F(T-t)  „ 

minimizes  it  e  Gu(t) 


subject  to  u(t)  e  U  , 


u  -k’  /  k’  k 

where  it  =  it  ,  it 

\  ’  n+1 


(5.42) 


The  problem  (5.42)  for  polyhedral  U  has  previously  been  shown  to  be 
a  parametric  linear  programming  problem  and  is  solvable  by  the  methods 
introduced  in  Chapter  IV. 

k+1  k+1 

The  generation  of  P  from  u  (t)  is 


F(T-t)  k+1., 
e  Gu  (t)  dt 


The  generalized  program  (5.39)  terminates  when 


k  /  P  /  1 
(n+1)  l  n+1 


for  some  iteration  k  of  (5.40).  We  know  that  the  minimum  value  of 

w  for  all  P  e  C  is  zero,  since  X.  e  R  ,  and 

l  T 

i* 

P  =  X  e  C  . 


mgam&v 


1 


There  fore 


i  *  +  — 

P  M  +  Iy  -  Iy  =  X. 


p,  y|,  y.  >  0,  has  a  solution  with  all  y*,  y^  =  0.  The  terminating 
condition  for  each  part  of  the  phase  I  procedure  terminates  a  general¬ 
ized  program  at  a  suboptimal  solution  with  the  objective  value  some 
specified  distance  from  its  optimal  value.  Thus  the  generalized  program 
terminates  in  a  finite  number  of  steps. 

Once  the  value  of  w  becomes  low  enough,  the  solution  to 
the  phase  I  procedure  for  each  must  be  recovered.  From  the  final 

solution,  for  each  phase,  let 


u1  (t)  =  u1(t)M.  and 


i*  V  r,i 

5  =  z  p  ^ 


for  each  i  =  0,  1,  ...,  n.  Also  note  that 


T 

■**  ■  I  ^  '  I  l 


eF(T  t)  G[u1(t) ]  dt  p. 


-f 

Jo 


F(T-t)  i*  ..  .. 

e  Gu  (t)  dt  . 


i*  i* 

Proposition  5.7.  The  set  of  vectors  P  and  controls  u  (t)  con¬ 
stitute  a  nondegenerate  feasible  solution  to  the  set  of  equations, 
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program  (5. 44) . 


Proof  of  Theorem  5.  2. 


Consider  the  master  program  at  the  k  stage,  as;  max  A 

M 

—1  ^  _»  _  _ 

subject  to  UqA  +  P  ,  +  P  +  Iy  -  Iy  -  Sv  =  0 


Mx  +  • * •  + 


Yf-  Yf >  Mi  > 


(5. 45) 


where 


P1  = 


1  1 


,  I  = 


-1  -1 


k  k 

Thus  A  =  -w  and  the  dual  variable  to  (5.45)  is 


k  k'  k  k 


it  =  I  it.,  it  ,  it  ,  ,  it  _ 
\  0  n+1  n+2 


By  using  the  duality  theorem  of  linear  programming  and  the  dual  to  (5.45) 


Also 


,  k  k  k 

A  -  nn+l  +  ”n+2  . 


0k  min  k' 

&  =P  £  C  ”  1 


min  k'  k 

:p  g  c  *  p  +  Vi 
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Since  S  is  always  in  the  basis  of  (5.45), 


jt  0=0  implies 


k'  .  k 

n  ("S)  =  "nn+2  • 


(5.46) 


By  hypothesis, 


„  0k  k 
0  <  5  +  w 


min  k'  k  k 

n  „  it  P  +  it  ,  +  w  , 
P  e  C  n+1 


_  _k  k  k'  k  k 

0  <  &  +  w  <  it  P+it  „  +  w 

-  n+1 


for  any  P  e  C  . 


_  k  ,  k  k  k 

Since  w  =  -A  =  -it  ,  -  it  _  , 

n+1  n+2 


_k  k  k„  k  ,  ,  ... 

0  <  5  +  w  <jtP-rr  _,  and  by  (5.46) 

—  n +6 


n  ..k  k  k'  k' 

0  <  5  +  w  <  it  P-n  S,  or 


_  ok  k  k'  ,  „ 

0  <  6  +  w  <  it  (P-S)  , 


for  all  P  . 


which  implies  that 


Thus,  for  any  P  e  C  , 


0  <  it  (P-S)  , 


it  ^  0  and 


P  -  S  ^  0  for  all  P. 
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Thus  there  is  no  admissible  control  function  u(t)  which  can  generate 
a  feasible  solution  to  the  control  problem. 


Q.  E.  D. 


However,  if  at  any  iteration  k,  w  =0,  the  current 

solution  is  a  feasible  solution  to  the  control  problem  and  phase  II 

(the  optimization  phase)  of  the  generalized  program  can  be  initiated. 

Since  this  phase  I  procedure,  unlike  the  previous  one,  is  not  necessar- 

k 

ily  a  finite  process,  the  optimization  phase  may  begin  when  w  <  t,  some 
small  positive  number  and  the  desired  final  point  not  precisely  attained. 
For  any  practical  control  problem,  when  feasibility  is  not  known,  a  point 
at  some  arbitrarily  small  distance  away  from  a  determined  fixed  point 
would  be  an  acceptable  terminal  point  for  the  control  problem.  Thus, 
the  phase  I  procedure  would  be  finite  even  when  feasibility  is  not 


assumed. 


For  the  variable  end  point  problem,  phase  I  procedures 


are  much  simpler.  For  example,  when  the  desired  final  region  is 

constrained  to  lie  in  some  r-neighborhood  about  a  determined  point  S, 
the  phase  I  procedures  outlined  previously  are  used  (r  <  p).  If  fea¬ 
sibility  is  assumed,  a  series  of  reachable  points  in  the  r-neighborhood 
should  be  chosen,  and  a  phase  I  procedure  identical  to  the  first  one 
discussed  in  this  paper  would  provide  a  nondegenerate  feasible  solution. 

If  a  nondegenerate  solution  is  not  necessary,  then  a  procedure  identical 
to  that  for  finding  the  existence  of  a  feasible  solution  can  be  used. 

In  this  case  S  is  allowed  to  be  the  right-hand  side  of  (5,44)  and  the 
algorithm  is  terminated  when  w  <  r,  as  the  following  theorem  points  out. 

k  * 

Theorem  5.3.  If  w  <  r,  then  the  solution  P  to  (5.44)  at  stage  k, 
which  produces  the  value  w  ,  is  a  vector  which  lies  in  an  r-neighbor- 
hood  about  S,  and  it  is  a  feasible  solution  for  the  control  problem. 
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Proof  of  Theorem  5.3. 


Since  w  <  r, 


n  n 


i=l  i=l 


y 

i=l 


thus 


+ 


Si)2  <  r- 


and 


* 


P 


Q.  E.D. 


A  general  phase  I  procedure  for  the  variable  end  point 
problem  can  be  used  to  determine  feasibility  as  well  as  to  find  a  fea¬ 
sible  solution  when  it  exists.  We  consider  the  problem 


n 

min 

w 

(J.  v 

i=l 


+ 

yi  + 


I 

i=l 


yi 


subject  to 


Pm  +iy 


+ 


iy  +  Sv  =  0 


M 


=  0 


v  =  1 


P  €  C 

S  e  <?  ,  or  equivalently,  (5.47) 


LHk.4kMk.-4k.lh.  lUl^C 


Theorem  5.4.  If  at  any  stage  k  of  the  initial  pliase  procedure  for  the 


variable  end  point  problem  w's  +  b^  +  >  0,  the  original  control 


problem  is  infeasible. 


Proof  of  Theorem  5. 4. 

— k'  /  k  k'  k  k 


_ k 1  /  |<  k 1  k  k  \ 

Letting  n  =  (ixQl  n  ,  nn+1>  nn+2  /  anci  1,emembering  from  the 


duality  theorem,  that 


k  .  k 
w  =  -A  = 


-It  ,  -  IT  n  , 

n+1  n+2 


then,  by  liypothesis, 


„  k  k  k 
0  <  b  +  '\  +  w 


min  — k'  P  min  — k' 
PeC^  +Se^n 


Substituting  in  the  above, 


J<  k  k  -k' 
0<b  +  A  +  w  <  it 


k  k 

-  IT  “It  „ 

n+1  n+2 


for  all  P  e  C  and  S  e  & 


„  k  k  k  k'  k'  k  k 

0  <  b  +  A  +  w  <  n  P+n  (-S)  +  it  +  itn+2 


hn+2  '  °r 


0  <  it  (P-S)  , 
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which  implies  that 


a  *  0 


(P-S)  ^  0 


for  all  P  c  C  and  S  t  A 


Therefore,  there  is  no  feasible  solution  to  the  control  problem. 


Q.  E.  D. 


Theorem  5.4  indicates  an  infeasibility  condition,  and  in 


the  next  theorem  (Theorem  5.5)  a  feasibility  condition  is  presented. 


Theorem  5.5.  If  at  any  iteration  k  the  solution  to  (5.48)  provides 


a  vector, 


>  x 


k  * 

and  a  value  w  <  r  so  that  S  is  an  interior  point  in  S  and  has 


an  r-neighborhood  surrounding  S  that  is  also  in  <$ ,  the  original  con¬ 


trol  problem  is  feasible  and  the  solution 


-i*  \'  i  k  r  k 

=  Z  Mi  Cm1 


being  a  solution  to 


(5. 48)] 


is  a  feasible,  reachable  point. 


Proof  of  Theorem  5.5. 


Since  w  <  r,  the  vector, 


v  i  k 

>  Pm 
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*  .-I*  •  -  ‘  • 


»  ■*-  ■*.  •*.  f 
•>  V  j-  v  ■ 


has  the  property 


Thus  the  point  P  is  in  an  r-neighborhood  about  S  implying  that 
*  * 

P  e  A.  Hence  the  control  generating  P  is  a  feasible  control. 

Q.  E.  D. 


2.  Generation  of  an  Optimal  Solution 

For  the  fixed  end  point  problem,  the  subproblem  for  phase  I 
methods  is  a  parametric  linear  program  of  the  type  discussed  in  Chapter 
IV.  For  some  variable  end  point  problems,  the  phase  I  procedure  has  an 
added  subproblem, 


min 
S  e  A 


k' 

it 


or 


min  k' 

S  e  A  "  n  (5.51) 

Thus  we  seek  a  vector  S  in  a  specified  set  A  that  minimizes  the  sum 


K'-v  ■ 

subject  to  the  constraint  that  S  e  A-  If  A  is  a  convex  polyhedral 
set,  this  problem  is  a  linear  program  that  must,  be  solved  once  for  each 
iteration  of  the  master  problem.  For  other  classes  of  A ,  the  sub¬ 
problem  depends  on  the  definition  of  A. 


jj 

i> 


While  discussing  the  optimization  phase  of  the  generalized 
programming  formulation  of  the  optimal  control  problem,  we  will  con¬ 
sider  only  the  fixed  end  point  problem. 

To  avoid  degeneracy,  it  is  desirable  to  start  the  optimization 
phase  with  a  set  of  n  +  1  vectors  P1  that  provide  a  nonsingular 
basis, 


ft: 


ip 

$ 

m 


r'J 

m 
dfci 

u  n 

m 


i 

rtf 

■$ 

>  V* 


P 

pi: 

km 


B  = 


-1  -2 
U  P  P 
0 


rn+1 


which  is  feasible  for  the  program, 


max  A  .  — 1 

i.  V  +  p  ui  * 

subject  to 


-n+1 
+  P  P 


n+L 


n+1 


=  S 


=  1 


Pi  >  0  • 


(5. 52) 


m 

rT 

% 

1  f (x, u  )  dt 

/n 

A- 

— i 

-/0 

-V 

where 

P  = 

A- 

i 

P 

_  _ 

where  uX(t)  t  U  generates  the  vector,  P1 ,  and 


S  = 


0 

s 


These  vectors  are  immediately  available  when  using  the  phase  I 
procedure  discussed  initially.  For  other  phase  T  procedures,  once  a 
feasible  solution  is  found  and  assuming  a  neighborhood  about  that  solu¬ 
tion  is  also  feasible,  the  initial  phase  I  procedure  may  be  used  by 
generating  the  right-hand  sides  in  a  similar  manner  about  the  known 
feasible  solution. 


Restating  the  fixed  end  point  optimal  control  problem  as 


x  =  Fx  +  Gu 


x(0)  =  0,  x(T)  =  S 


x  e  En  ,  u(t)  e  U  c  Em  »  Vt  , 


where 


U  =  { u  Au  >  b)  , 


J  -  / 


f(x,u)  dt  , 


where 


fox  +  gou 


f(X'U)  =  <  foX  +  g0U  +  Z  lUil 


f^x  +  g^u  +  u’Qu 


we  can  define  F  and  G,  as  before. 

When  a  set  of  vector  control  functions  u*(t)  is  given  from 
the  phase  I  procedure,  the  vectors  P*  must  be  generated. 


c 


+  g^u)  dt  +  I  f(u)  dt 


T  —  T 

Px=  f  eF(T-t)  Gu( t)  dt  +  f  f (u)  dt  UQ  . 
•'O 


Since  the  matrix  e  must  be  provided  for  the  phase  I  method,  the 


matrix, 
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is  easily  shown  to  be 


by  considering  the  system 


y°  =  v  x  e  e 


x  (t)  =  Fx 


where  x(t)  =eF^  x(t),  and 


Xq  ( t )  =  f^xCt)  =  f^e1^1  ^  x(t)  . 


Le  1 1 i ng  x  = 


F(t-t)  - 
e  x(t)  = 


,,  F(t-t) 

fie 


m-i) 


x0(t> 


From  (5.52)  we  see  that  the  linear  equations  in  p ^  always 

insure  that  a  feasible  solution  exists  for  the  control  problem.  The 

simplex  method,  when  applied  to  the  master  problem,  maintains  primal 

feasibility  at  all  times  even  when  augmenting  the  linear  equations  with 
— k+1 

a  vector  P  .  Thus  at  any  time  in  the  execution  of  the  optimization 
phase,  a  feasible  solution  is  available  from  the  current  basis.  Also 
as  will  be  shown,  a  bound  on  the  optimal  solution  is  provided  at  each 
stage. 


SEL-68 -085 


f.  •*.  *|VV . 

•»  •  *  V  *  -  ■ 
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-  r  %  v  •  •  •  "  >•*  •  *  . 

■  •  •  .>  k,  r  - >  * >  -  *>  "  ►  *  %»>  •  *  •  *  W  ■ 
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In  the  following  let  J  =  -A  ,  where  A  is  the  objective 

th 

value  of  the  master  problem  in  the  k  iteration. 

th 

Theorem  5,6.  During  any  k  iteration  of  the  optimization  phase  of  the 

* 

generalized  program,  the  optimal  value  of  the  cost  function  J(u  ) 
satisfies  the  following  inequalities  [2]: 

Jk(u)  +  &k  <  J(u*>  <  Jk(u)  <  Jk  =  -Ak  ,  (5.53) 

where  k 

G(t)  =  Mx(t)  . 

i=l 


Proof  of  Theorem  5.6. 

Consider  the  equivalent  linear  program,  as  before, 


max  A 

M 

V  +  p  +  +  ‘ 


.  .+PkMl  -  SV  =  0 
Kk 


Ml  +  M2  +  •  •  •  +  Pk 


v  =  1 


u.  >  0  . 

- 


(5. 54) 


The  solution  to  (5.54)  is  A  and  by  the  dual  theorem, 


..  k  k  k 

A  =  It  ,  4  It  _  , 

n+1  n+2 

where  it  =  ( it  ,  it  ,  it  „  ,  it  „  is  the  dual  variable  to  (5.54).  Since 
,  \  0  n+1  n+2/ 

it  =  1,  the  subproblem  has  the  solution, 


J*  min  )  t/  \ .  k '  k 

6  =  ,  .  <  J(u)  +  it  P  +  it 

u ( t )  e  U  I  n+1 
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Thus  for  the  value  u  , 


k  *  k'  *  k 

b  <  J(u  )  +  it  P  +  it  ,  ,  and 

n+1 


k  k  k  *  k'  *  k  k  k 

A  +  6  =  J  +  b  <  J  ( U  )  +  It  P  +  it  -it  .-it  „  , 

—  n+1  n+1  n+2 


or 


Jk  + 


6k  <  J(u*)  +  itk' 


* 

P  - 


n+2 


Since 


-k’ 

it 


0 

-S 

0 

1 


k*  „  k 

=  -*t  S+IT  =0, 
n+2  ’ 


k  k  *  k'  * 

J  +  0  <  J(u  )  +  it  (P  -  S)  . 


*  *  * 

If  u  is  the  optimal  u,  then  P  is  feasible  and  P  =  S.  Thus, 


J*  +  5k  <  J(u  )  <  Jk(u),  where  Jk(u) 


is  the  current  solution  and  u  is  a  feasible  control,  and  the  right- 

* 


hand  inequality  follows  immediately  since  J(u  )  is  the  minimum  cost 
Since  J(u)  is  convex  in  u, 


Jk(u)  <  Jk  =  ^  p. JCu1)  . 


i=l 


Therefore 


k./v.  k  k  ^k  .  k,/sN  k 

J  (u)  +  b  <  J  +  b  <  J(u  )  <  J  (u)  <  J 


(5. 
Q.  E 
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Corollary  5.1.  When  =  0,  Jk(u)  =  Jk  -  J(u  ). 


Proof  of  Corollary  5.1. 


The  proof  follows  from  (5.55),  since  equality  holds  throughout. 


Q.  E.  D. 


Note  that  the  value,  A  ,  of  the  master  problem  is  an  approximation  to 
the  current  solution  at  iteration  k.  At  any  stage,  the  solution  de¬ 
fined  by 


V  k  i  .  . 
2  "1“  (,) 


has  a  cost  J(u)  which  by  convexity  is  smaller  than  J  . 

Although  6  does  not  necessarily  increase  monotonj.cally  to 
0,  it  does  so  for  a  subsequence  of  k.  Thus  the  best  bound  from  pre¬ 
vious  iterations  should  be  retained  until  a  better  bound  is  attained. 
The  current  value  (at  iteration  k  of  the  generalized  program)  of  6 
may  be  used  to  provide  a  stopping  condition  for  termination  of  the  op¬ 
timization  phase.  By  observing  the  value 


6k/Jk  , 


we  can  determine  the  maximum  percentage  by  which  the  objective  function 
can  decrease  for  the  optimal  solution,  and  we  are  assured  that  the  cur¬ 
rent  solution  is  feasible  to  the  original  control  problem. 

We  now  present  a  flow  chart  of  the  generalized  programming 
solution  to  the  optimal  control  problem.  We  will  use  the  fixed  end 
point  problem  for  an  example,  assume  a  p-neighborhood  about  S  is  in 
R^,  and  apply  the  long  form  of  the  phase  I  procedure. 


\ 


Let  P1  = 

l 

J 

u^t)  =  V  uV 

Z_, 

fr 

j 

for  all  i 

i- I 

o~ 

ii 

ph.'.se  II 


1*1 


Set  k  =  n 


I* 


max  A 

P 

-  0 

1 

k  _ 

Subject  to 

A  +  Vo 

+  PqM!  + 

...  +  PoM](  =  0 

0 

1 

k 

P  u 

po 

+  P  4l  + 

...  +  P  Mk  =  s 

P0 

+  p1  + 

ii 

=L 

+ 

Pi  n  0 

i  =  0,  1,  .  .  .  ,  k 

Calculate  rr  -  dual  to  linear  program 


Is  Phase 


F  =  F,  f(u)  =  0 
G*  =  G,9  =0 


0 

f  (u)  = 

u'Qu 

Y“.' 

F  *  =  F 

,  C*  =  C,  6  =  1 

* 

Find  n  (t) 

* 

-k' 
min  n 

F  (T-t) 
e 

G  u(t)  +  f(u) 

Subject  to 

Au  > 

b 

t  £[0, 

T] 

T  * 

t 

■J, 

r  F  (T-i )  * 

e  1  G  u(l)  dt  + 

D 

f  f(u  )  dt  UQ 

'o 

Is  6=0? 


Is  b  >  0  ? 


Is  ixL<^ 

speci  fiecl  £  $ 


*  ! 

i  Y  1 

u  ( t)  =  are  i 
1 

\1  p 
'  i 

is  optimal 

solution 

The  inputs  to  t ho  flow  chart  are 


1 


System  matrices  F,  G 

Loss  functions,  f(x,u)  =  fg(x,u)  +  f(u) 
where  ^(x.u)  =  f^x  +  g^u 

Parameter,  p. 

Final  time  T  [assuming  initial  time  0] 


Final  state  S  or  S 


Polyhedral  matrix,  A 


-  so] 


Right-hand  side  vector,  b 


Unit  vectors,  U  ,  e^  e2>  ...,  e^+1 

8o  =  0 

the  vector  1  . 


To  retain  the  continuous-time  aspects  of  the  control  problems, 

this  algorithm  requires  explicit  knowledge  or  availability  of  the  matrix 
Ft 

e  ,  and  its  time  derivatives. 

It  is  well  known  [4]  that  the  components  of  this  (n  x  n) 

matrix  can  be  expressed  as  polynomials  of  an  order  less  than  or  equal  to 

Ft 

n  with  an  exponential  multiplying  factor.  The  knowledge  of  e  is 
required  for  determining  the  functions  7 ( t ) ,  used  in  the  parametric 
programming  subproblem,  and  for  determining  the  vectors  P,  given  a 
control  function  u(t),  over  an  interval. 

We  can  express  the  fundamental  matrix  as 


.  I  V„ 

k=0 


where  m  is  the  degree  of  the  minimal  polynomial  of  F.  Note  that 
m  <  n,  if  F  is  (n  X  n) .  The  algebraic  equations  determining  ft 
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where  O'  is  an  m-vector  and  C  is  a  vector  with  elements  of  the  form 
f  sl<t 

t  e  ,  k  =  1,  .  .  .  ,  0;  f  =  0,  .  .  .  ,  m  -  1,  where  the  s' s  are  the 

K  K 

eigenvalues  of  F  and  is  the  multiplicity  of  the  eigenvalue,  s  . 

M  can  be  shown  to  be  nonsingular.  Thus  Of  is  composed  of  linear  com- 

K 

binations  of  the  elements  of  f<(t),  which  are  themselves  members  of 
the  class  of  solutions  to  homogeneous,  constant  coefficient,  linear  dif¬ 
ferential  equations.  Thus  any  vector 


l)  =  ,'eF<T-t>  0  , 


where  tt  is  a  real  vector  and  G  is  a  real  matrix,  has  components 
which  are  members  of  the  class  of  solutions  to  the  homogeneous,  constant 
coefficient,  linear  differential  equations. 

As  will  be  proven  in  the  next  chapter,  the  functions  generated 
for  the  linear  loss  case,  minimum  fuel  problem,  and  minimum  time  problem 
are  piecewise  constant  functions.  Also,  for  the  quadratic  loss  in  con¬ 
trol  problem,  the  generated  function  u(t)  is  shown  to  be  expressed  as 

a  linear  combination  of  the  Qf/s  for  a  finite  interval  of  t.  Thus  the 

k 

components  of  the  vector  integral 


"2  er(T-t)  Gu( t)  dt 


can  be  represented  by  a  sum  of  integrals  of  the  form, 


2  e  st  , 
ate  d  t  , 

s 


which  when  integrated,  is  equivalent  to 


t«eSt  *2  f 

st,/ 


1  t. 


2  e  e-i  st 

a  —  t  e  dt 
s  s 


(5. 56) 
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1' 

V? 


r>: 


fvy 
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O 


K» 


r- 


Ft 


It  should  be  noted  that  all  components  of  e  will  be  real,  even  for 


complex  eigenvalues,  since  cancellations  of  the  complex  part  occur. 

Ft 


Since  the  values  of  e  and  its  first  n  -  1  derivatives 
are  needed  at  only  a  finite  number  of  points  (undetermined),  these  val¬ 
ues  may  be  provided  by  an  analog  computer.  This  suggests  (but  not  re- 
strictively)  the  use  of  hybrid  computers  for  the  algorithm.  The  analog 
computer  could  be  used  to  supply  values  of  /(t)  at  specified  instants 
and  to  compute  the  vectors,  P[u(t)],  while  a  digital  computer  could 
be  used  to  solve  the  linear  and  quadratic  programming  problems  in  the 
master  and  subproblems. 

To  show  that  the  algorithm  is  computationally  feasible,  we  will 
show  that  each  step  or  block  in  the  flow  chart  is  solvable  by  a  finite 
number  of  iterations.  Although  convergence  of  generalized  programs  may 
be  an  infinite  process,  a  suboptimal  solution  as  close  as  desired  to  the 
optimal  solution  is  achievable  in  a  finite  number  of  iterations  of  the 
subproblem.  However,  when  the  reachable  set  is  a  polyhedron,  the  gen¬ 
eralized  program  converges  in  a  finite  number  of  iterations  of  the  sub¬ 
problem.  The  number  of  these  iterations  is  less  than  or  equal  to  the 
number  of  extreme  points  of  the  polyhedron. 

To  demonstrate  the  finiteness  of  the  executions  at  each  stage 
in  the  algorithm,  we  will  show  finiteness  for  each  block  of  the  flow 
chart  for  the  basic  algorithm  (note  that  the  block  numbers  designated 
coinc.de  with  those  on  the  flow  chart.)  It  is  also  noted  where  an  ana¬ 
log  computer  may  be  substituted  when  hybrid  computations  are  desired. 


Block  1 


If  the  matrix  e 


Ft 


is  not  available  as  an  input,  its  deter¬ 


mination  may  be  obtained  using  an  eigenvalue  analysis  of  the  matrix  F; 


routines  of  this  nature  are  available.  After  the  eigenvalue  analysis, 


F(T-t) 

a  set  of  linear  equations  must  be  solved  to  find  e  in  terms  of 


a  finite  sum  of  multiples  of  F.  The  analog  computer  may  be  utilized 

Ft 


for  determination  of  e 


Block  2  :  The  extreme  points  of  a  simplex,  in  the  reachable  set  surround¬ 


ing  the  desired  point  S,  is  a  computation  involving  addition  and  sub¬ 
traction  of  the  available  vectors,  some  of  which  are  unit  vectors. 


L>  • 

& 

VS 

ft 

ill 
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Block  5  :  The  solution  to  the  phase  I  master  problem  at  any  k  for  any 
right-hand  side  is  a  linear  program  that  has  an  initial  basic  solution 
immediately  available,  i.e., 


p.  =0,  Vi 
1 


+ 

yi 

=0,  yt  =  -S.  , 

for 

s 

i 

< 

0 

+ 

yi 

*  Si'  =  °  ' 

for 

S 

i 

> 

0  . 

The  number  of  rows  in  this  linear  program  is  n  +  1  for  any  stage  k, 
even  though  the  number  of  columns  is  variable  but  always  finite. 

Block  6  :  Additional  columns  are  added  to  the  master  program  of  phase 

I  until  the  value  of  w  is  less  than  a  required  strictly  positive  num¬ 
ber.  Since  the  minimum  value  of  w  is  zero  and  since  w  decreases 
monotonically  and  strictly  decreases  on  a  subsequence  of  iterations,  for 
some  specified  positive  number,  the  value  of  w  will  be  smaller  than 
this  number  after  a  finite  number  of  iterations.  This  is  a  basic  result 
of  generalized  programming  problems  and  shows  only  a  finite  number  of  col 
umns  are  used  for  the  master  program. 

Block  7  :  The  calculation  of  the  dual  variable  of  the  linear  program  in 
the  master  problem  is  a  result  of  the  solution  procedure  for  the  linear 
program  and  requires  little  or  no  additional  computation. 

Block  8  :  A  vector  addition  provides  the  column  vectors  to  be  used  in 
the  master  program  after  the  phase  I  procedure  is  completed. 

Blocks  9  and  13  :  The  determination  of  the  vector  P  is  achieved  by 
intregation.  However,  due  to  the  structure  of  the  integrand,  special¬ 
ized  (finite  and  exact)  integration  schemes  are  possible.  The  integral 
is  broken  into  a  finite  sum  of  definite  integrals  (corresponding  to  a 
finite  number  of  switching  points)  whose  end  points  are  calculated  by 
the  analog  computer  or  by  formula  substitution  as  suggested  by  Eq.  (5.56) 


Clock  11 :  The  master  program  for  the  optimization  phase  is  a  linear 
program  with  a  fixed  number  of  rows  (at  most  n  +  3)  and  a  variable 
number  of  columns.  Although  the  number  of  columns  may  be  infinite,  for 
any  practical  problem  and  within  limits  of  the  computer's  accuracy,  no 
more  than  a  finite  number  of  columns  are  generated  before  achieving  a 
solution,  indistinguishable  (within  computer  accuracy)  from  the  optimal 
solution.  The  simplex  method  should  be  used  with  the  starting  solution 
to  each  interation  being  the  final  solution  of  the  previous  iteration. 
The  first  vector  to  be  added  to  the  basis  is  the  vector  generated  from 
the  subproblem  (if  optimality  has  not  already  been  achieved). 

Block  12:  The  solution  of  the  parametric  programming  problem  is  dis¬ 
cussed  in  Chapter  IV.  The  solution  has  a  finite  number  of  executions; 
cycling  is  avoided  due  to  the  lexicographic  ordering  rules  and  normal 
degeneracy  perturbation  techniques  available  for  linear  programming 
codes  and  complementary  pivot  theory  methods. 

All  other  steps  are  either  logical  programming  steps  or  simple 
calculations.  Thus  since  each  step  requires  a  finite  number  of  execu¬ 
tions,  each  iteration  of  the  master  problem  and  its  corresponding  sub¬ 
problem  (of  phases  I  and  II)  require  a  finite  number  of  executions. 

The  solution  of  phase  I  is  finite  since  only  a  finite  number 
of  columns  must  be  generated  for  the  n  +  1  generalized  programs  used 
for  the  solution  to  each  step  of  phase  I.  The  algorithm  may  be  termi¬ 
nated  at  any  stage  in  phase  II  yielding  a  feasible  control  with  a  bound 
on  how  much  its  objective  value  can  differ  from  the  optimal  objective 


Chapter  VI 


itSii 


M  ■  > 


fin 
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RELATION  OF  GENERALIZED  PROGRAMMING 
TO  CONTROL  THEORY 

In  this  chapter,  the  relationship  between  the  necessary  conditions 
of  the  generalized  programming  formulation  and  Pontryagin's  necessary 
conditions  for  the  optimal  control  problem  is  discussed.  The  character¬ 
istics  of  optimal  controls  for  the  various  classes  of  control  problems 
are  also  discussed. 


ti 


& 


A.  Relation  to  Pontryagin's  Necessary  Conditions 

The  relationship  between  the  generalized  programming  optimality 
conditions  and  Poncryagin's  necessary  conditions  is  used  to  show  how  a 
solution  to  the  generalized  program  can  be  an  optimal  solution  to  the 
control  problem.  The  following  class  of  problems  (discussed  in  the  pre¬ 
vious  chapter)  are  considered: 


n  r,m 

x  e  E  ,  u  e  E 


x  =  Fc  +  Gu 


T 

min  J  =  L  jf^x  +  gpU  +  f(u)j  dt 


where  f(u)  = 


-S'-. 


x(0)  =  0 


x(T)  =  S 


u(t)  e  U  =  {u|  Au  >  b} 


If  we  lev  Xq  =  f^x  +  g^u  +  f(u),  then 


,  Pol 

*-UJ 


=  Fx  +  Gu  +  f(u)  IL 
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By  using  the  previously  presented  notation,  the  Hamiltonian  can  be  defined 


as 


H  =  \Jr'Fx  +  \Jr'Gu  +  \Jr' f(u)  Uq 


x  =  H—  ,  and 

'if 


.  (t) 

'  ♦jCt) " 

where  \)r(t)  = 

o 

and  \|r(t)  = 

• 

1 

V* 

\Jr  ( t) 

L  n  J 

(6.1) 

(6.2) 

(6.  3) 


Pontryagin's  maximum  principle  states  that  (6.2)  and  (6.3)  must  hold 

•Hr 

while  the  optimum  u  (t)  satisfies 


_  * 


H<¥»  x,  u  )  =  u(^Pe  u  H(t-  x-  u) 


(6.4) 


for  a  given  \Jr,  x,  or 


H(\|r,  x,  u  )  >  x,  u)  ,  all  u ( t)  e  U 


(6.5) 


Let  the  optimal  dual  solution  to  the  generalized  program, 


be 


and  let 


max  A 
Per 

subject  to  UqA  +  Pp  =  S 

M  =  1 
P  e  C  , 


n 


* 

JT  ) 
n+1 


\|r(  t) 


F(T-t) 


(6.6) 
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where  it  =  (it  ,  iO  .  Thus  it  is  obvious  that  4r(  t) ,  as  defined, 

provides  a  solution  to  (6.3),  To  show  that  this  solution  is  non-trivial, 

it  is  sufficient  to  show  that  (t)  ^  0.  From  the  results  obtained  in 

F(T-t) 

Chapter  V,  the  first  column  of  e  is  the  unit  vector  U^.  Thus, 

^Q(t)  =  ,  for  all  t  .  (6.7) 

We  also  know,  from  generalized  programming,  that 

- 1  ■ 

it*  =  1  and  ( t)  =  -1,  for  all  t. 

•  ■  y. 

it  is  the  dual  solution  to  the  generalized  program,  it 


P  e  C  , 

or,  equivalently, 


mm 
P  e  C 


P 

P 

IT 

1 

<  IT 

_  1 

therefore, 

Since 

satisfies 


mm 


.*» 


u(t)  e  U 


P(u) 

<  IT 

P(u) 

1 

1 

Vu(t)  e  U  . 


(6.8) 


The  above  inequality  is  equivalent  to  the  subproblem  of  the  generalized 
—•X* 

program,  when  it  is  the  current  dual  variable.  Equation  (6.8)  may  be 
restated  as 

it  f(u  )  +  rt  e  ^  Uu  (t) 


*  ~*'  F(T-t)  — 

<  7t  f(u)  +  it  e  Gu(t) 

-  0 


for  all  u(t)  e  U,  and 
t  e  [ 0,  T]  , 
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or 


*  *  *•  FfT-t')  -  * 

-IT  f(u  )  -  IT  e  K  Gu  (t) 


*  *1  _ 

>  — iTq f ( u)  -  jT  e  Gu( t)  . 


By  expressing  tlie  above  in  terms  of  \jT( t)  and  by  using  Eqs,  (6.6)  and 
(6.7),  the  inequality  is  equivalent  to  (6.5)  or  Pontryagin's  maximum 
principle.  Thus  the  generalized  programming  necessary  conditions  are 
equivalent  to  Pontryagin's  necessary  conditions  for  the  same  problem. 

To  complete  the  analogy  between  Pontryagin's  necessary  conditions 
and  the  generalized  programming  optimality  conditions  for  control  prob¬ 
lems,  we  include  here  a  discussion  of  the  transversality  conditions  for 
sets  3  which  are  convex  smooth  manifolds. 

r^J  *X*  - 

Notice  that  the  vector  it  is  equivalent  to  the  vector  iJi(T)  . 

Also  an  optimality  condition  for  free  (initial)  final  point  problems 
[as  shown  in  Eq.  (5.47)]  is 


S 

IT 


>  0  , 


for  all  S  e  ij 


We  also  note  that  some  S  e  S  has  the  property 


*'  *  * 

IT  S  =  -IT  , 
n+2 


since  there  must  be  some  vector  S  in  the  basis  of  the  expanded  master 
linear  program. 

These  conditions  represent  a  halfspace  with  the  hyperplane  defining 

_■ fl¬ 
it  as  being  represented  by  the  vector  it  .  This  hyperplane  is  a  sup- 

*  *'  *  * 

porting  hyperplane  to  <4  at  some  S  ,  since  it  s  *  1  ies 

completely  in  one  halfspace  of  the  hyperplane.  The  hyperplane  is  also 
a  tangent  plane  to  the  manifold  &  (when  S  is  a  manifold).  These 
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conditions  represent  the  fact  that  \}r(t)  =  -jt  is  orthogonal  to  the 

* 

tangent  hyperplane  of  ^  at  S  .  This  is  precisely  the  transversality 
condition  described  in  Chapter  II. 

To  show  that  a  solution  to  the  generalized  programming  problem  is 
also  a  solution  to  the  continuous-time  optimal  control  problem  (for 
fixed  end  points),  it  is  assumed  that  we  have  a  finite  set  of  vectors 
P1,  so  that 


*• 


TT 


and 


*' 


rr 


all  P  €  C  . 


Also 


=  S 


=  1 


Mi  >  0 


(6.9) 


* 

has  a  solution,  p.  . 


Theorem  6.1.  The  solution 


* 

u  (t) 


* 

Mi 


t  e  [0, T] 


is  an  optimal  solution  for  the  control  problem. 


Proof  of  Theorem  6.1. 


We  know 


F(T-t) 

\j/(t)  =  -it  e 


satisfies  (6.3)  and  \|r^ ( t )  =  -1  satisfies  \|i  (t)  <0,  t  e  [C,T] 


*  F(T-t)  —  * 

f(u  )  +  it  e  V  Gu  (t) 


v  ’  *  i 

=  f  >  u.u  a) 


F(T-t)  -  V  *  i  .  . 

+  it  e  G  >p.u  (t)  , 


and,  from  convexity, 


f(u*)  <  f  ^  p*u1(t)  <  ^T^Ku1)  . 

_  _J 


Since  D  =0,  from  generalized  programming  necessary  conditions 


*  FfT-t)  —  *  * 

0  =  f(u  )  +  it  e  Gu  (t)  +  n 

n+1 


\  '  *  i  V  F(T-t)  —  i  * 

<  >  |.i .  f(u  )  +  >  I, .  it  e  Gu  (t)  +  \  |j . 

-  zL  1  /-  i  Z,  1 


*  * 
it 

i  n+1 


=  \  Ll  It  =  0  . 


Since  it  „  is  constant  and  since 
n+1 


>  0  ,  VP  e  C  , 
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.<*  V  "J*»  *  py  •  ».  +y  -JL  *  v  flrtH  -y  -  -  *J-  ry  ~r,  r 


*  ~*'  F(T-t)  —  *  ~#'  FfT-t')  — 

f(u  )  +  Tt  e  W  '  Gu  (t)  <  f(u)  +  *  e  ;  Gu(t)  , 


Vu(t)  e  U  , 


*  F(T-t)  -  *  FfT-t)  — 

-  f(u  )  -  it  e  Gu  (t)  >  -f(u)  -  it  e  Gu(t), 


u(t)  e  U  , 


t  e  [0,  T]  , 


which  is  equivalent  to  the  maximum  principle.  Thus  Pontryagin’s  neces¬ 
sary  conditions  for  optimality  are  satisfied  by  the  solution  to  the  gen¬ 
eralized  programming  problem.  Since  the  set  of  equations  (6.9)  has  a 

* 

solution,  the  control  u  (t)  is  a  feasible  control.  It  remains  to  be 


shown  that 


J(u  )  <  J(u)  , 


Vu(t)  e  U  , 


which  was  shown  to  be  a  result  of  6=0,  in  Theorem  5.6.  Thus  the 


control  u  (t)  is  an  optimal  control  for  the  continuous-time  control 


problem. 


Q.E.D. 


We  will  now  show  that  given  tt  ,  an  optimal  dual  solution,  a  finite 


set  of  vectors  P  can  be  found  to  provide  a  feasible  solution  to  the 


set  of  equations  (6.9).  This  is  done  for  three  cases,  the  quadratic 


loss  ' n  control  with  positive  definite  matrix  Q,  the  linear  cases 


satisfying  Pontryagin’s  general  position  condition,  and  finally,  the 


linear  cases  not  satisfying  Pontryagin’ s  genera]  position  condition. 


Theorem  6.2.  If  the  quadratic  lo '  j  control  problem  with  positive  def¬ 


inite  Q  has  a  feasible  solution,  then  u  (t),  which  provides  a 


solution  to 
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=  0  , 


is  an  optimal  control  for  the  control  problem. 


Proof  of  Theorem  6.2. 


Let  u  (t)  be  a  control  satisfying 


=  0  , 


tlien  u  (t)  solves 


min  n(t)'  Qu(t)  +  /  (t)'  u(t) 


u(t)  t  U  , 


t  e  [0,t]  , 


*  _* 

where  /  (t)  is  generated  by  n  .  The  quadratic  program  with  Q  posi¬ 


tive  definite  has  a  unique  solution  at  each  t.  Thus  t|lere  ts  no  other 


u(t)  satisfying 


By  the  feasibility  assumption  and  by  U  being  a  compact  set,  an  optimal 


control  exists  and  must  satisfy  tiie  necessary  conditions.  Since  u  ( t) 


is  the  only  control  satisfying  the  necessary  conditions  for  optimality, 


it  must  be  the  optimal  control. 


Q.  E.  D. 


Theorem  6.3.  For  feasible  linear  control  problems  (including  minimum 
fuel  and  minimum  time)  where  F,  G,  and  U  satisfy  the  general 


position  condition  of  Pontryagin,  the  control  u  (t)  satisfying 
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_*• 

IT 


P 


0 


is  an  optimal  control. 


Proof  of  Theorem  6.3. 


Since  a  feasible  control  exists  and  since  U  is  compact,  an  opti¬ 


mal  control  u  (t)  must  exist  and  satisfy  the  necessary  conditions  for 
optimality.  From  the  results  of  Pontryagin,  when  the  general  position 


condition  holds, 


min  — 
U ( t)  €  u  * 


(6.10) 


has  a  unique  solution  [of  u(t>],  except  on  a  set  of  measure  zero  , 


which  we  call  u  (t).  Thus  u  (t)  is  the  only  function  satisfying  the 


necessary  conditions,  and 


*  ** 

U  (t)  5  U  (t) 


Q.  E.  D. 


When  the  general  position  condition  is  not  satisfied  by  F,  G, 
and  U,  the  solution  to  (6.10)  is  not  necessarily  unique  over  a  set 


of  positive  measure.  However,  since  (6.10)  must  be  satisfied  (be¬ 


cause  it  is  a  necessary  condition),  only  its  solutions  need  be  investi¬ 


gated  to  produce  the  optimal  control.  This  is  true,  since  the  problem 


is  feasible,  U  is  compact,  and  an  optimal  control  exists.  By  the 


theory  of  generalized  programming,  any  solution  to  (6.10)  which  is  fea¬ 
sible  for  (6.9)  is  an  optimal  solution,  as  shown  by  Theorem  6.1. 


Proposition  6.1.  There  are  a  finite  number  of  distinct  solutions  to 


(6.10) . 


SEL-68-085 


Proof  of  Proposition  6.1. 


It  lias  been  shown  in  Chapter  IV  that  an  upper  bound  exists  on  the 
number  of  possible  switching  points  of  any  solution  to  the  parametric 
programming  problem  (6.10),  for  any  value  of  it  for  a  finite  interval 
of  t.  These  poiMs  are  fixed  (given  l7*),  and  any  solution  to  (6.10) 
remains  constant  between  any  neighboring  pair  of  switching  points.  There 
are  a  finite  number  of  possible  solutions  for  (6.10),  between  such 
switching  points  (due  to  a  finite  number  of  bases).  Thus  there  are  a 
finite  number  of  distinct  solutions  to  (6.10). 

Q.  E.  D. 

Proposition  6.2.  The  set  of  points  P,  generated  by  (6.10),  are  extreme 
points  of  a  convex  (bounded)  polyhedron  of  all  P  satisfying 

=  0  .  (6.11) 

Proof  of  Proposition  6.2. 

By  the  minimization  procedure  of  (6.10),  extreme  points  are  gen¬ 
erated.  Also  by  the  homogeneity  of  (6.11),  any  convex  combination  of 
the  finite  number  of  extreme  points  satisfy  (6.10).  Thus  the  points 
generated  by  (6.10)  are  extreme  points  of  the  convex  polyhedral  set 
containing  the  solutions  to  (6.11). 

Q.  E.  D. 

Theorem  6.4.  For  linear  loss  problems  not  satisfying  the  general  posi¬ 
tion  condition,  a  finite  set  of  vectors  P1  can  be  found  to  pro¬ 
vide  an  optimal  solution  to  the  generalized  programming  problem. 

Proof  of  Theorem  6.4. 

The  optimal  solution  satisfies 

it 
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s 


hence  we  have  shown  that  a  finite  number  of  vectors  P  form  the  extreme 
points  of  all  vectors  satisfying  the  Eq.  (6.11).  Since  an  optimal  solu¬ 
tion  exists,  it  must  satisfy  (6,10),  and  therefore,  it  must  be  a  convex 
combination  of  all  extreme  points  to  the  set  of  vectors  satisfying  (6.11). 
It  can  be  shown  that  an  optimal  control  vector  P  must  be  a  combination 
of  at  most  n  +  1  extreme  vectors  P1  satisfying  (6.10).  Thus,  the 
finite  set  of  extreme  vectors  satisfying  (6.10)  can  be  generated  a  id  must 
include,  in  its  convex  hull,  a  feasible  solution  to  (6.9).  Therefore, 
the  optimal  solution  is  determined  by  a  finite  set  of  vectors  P3 . 

Q.  E.  D. 


Let  be  the  solution  to 

=  S 


=  1  ,  (6.12) 

where  the  P~  are  the  extreme  vectors  of  (6.10)  or  (6.11).  Such  a 
solution  exists  from  Theorem  6.4. 

*  V*  *  i 

Theorem  6.5.  u  (t)  =  \  p^u  (t)  , 

i=l 

where  u1(t)  generates  P1 ,  is  an  optimal  solution  to  the 
control  problem. 

Proof  of  Theorem  6.5. 

* 

u  (t)  provides  a  feasible  solution  by  virtue  of  (6.12)  and  satis¬ 
fies  the  necessary  conditions  by  construction;  by  Theorem  6.1,  the  cost 
function  is  minimal  over  all  feasible  controls. 

Q.  E.  D. 


i  *—1  *-k 

U0A  +  *  ■  ■  ■  +  >'kP 


y 


i=i 
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13.  Characteristics  of  Linear  Loss  Optimal  Controls 


Without  assuming  a  general  position  condition,  we  will  show  that  an 
optimal  solution  to  t lie  control  variable  for  linear  loss  functionals, 
including  minimal  fuel  and  minimum  time  problems,  is  a  piecewise  con¬ 
stant  function  with  a  finite  number  of  points  of  discontinuity  for  any 
finite  interval  of  time.  Thus,  since  we  are  considering  finite  horizon 
problems  (T  <  <*) ,  an  optimal  control  is  a  piecewise  constant  vector 
function  with  a  finite  number  of  switching  points. 

The  solution  to  the  parametric  linear  program  is  observed  as  being 
a  vector  control  function  that  is  piecewise  constant  and  has  a  finite 
number  of  switching  points.  Thus  any  vector  P1,  generated  by  the 
subproblem,  has  the  same  property  for  its  generating  control  function. 
Since  generalized  programming  problems  are  linear  programs  in  the  master 
problem,  and  since  the  number  of  rows  in  the  linear  program  is  less  than 
or  equal  to  (n  +  3)  ,  the  number  of  columns  P1  in  any  solution  is  at 
most  (n  +  1)  for  phases  I  and  II  of  the  algorithm. 


Proposition  6,3.  The  columns  P  ,  generated  for  the  optimization  phase 
of  the  algorithm  by  the  phase  I  procedure,  are  generated  by  control 
vectors  that  are  piecewise  constant  with  a  finite  number  of  dis¬ 
continuities. 


Proof  of  Proposition  6,3. 

In  general,  the  n  +  1  columns  P1,  for  i  =  0,  1,  ...,  n  gen¬ 

erated  for  an  initial  feasible  solution  to  the  control  problem,  are  gen¬ 
erated  from  the  phase  I  algorithm  for  n  +  1  right-hand  sides.  Thus 
each  P1  is  generated  by  a  set  of  at  most  n  +  1  vectors  P,  and 
each  in  turn  is  generated  by  a  piecewise  constant  control  function  with 
a  finite  number  of  switchings.  If  the  maximum  number  of  switchings  for 
any  control  function  generated  by  the  subproblem  is  M  (M  <  °o) ,  each 
P1  has  a  control  with  at  most  (n  +  1)M  switchings,  since  these 
controls  are  generated  by  summing  n  +  1  control  functions  with  at 
most  M  switchings  each. 
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Each  new  column  (after  feasibility  is  attained)  is  generated  by  a 
piecewise  constant  control  with  at  most  M  switchings.  Thus  each 
column  P1,  in  the  master  problem  of  phase  II,  has  at  most  (n  +  1)M 


switchings  in  control. 


Q.  E.  D. 


Proposition  6.4.  At  any  stage  in  the  iterative  process  of  the  generalized 
program,  the  current  control  solution  is  a  piecewise  constant  func¬ 
tion  with  a  finite  number  of  discontinuities  and  has  an  objective 
value  within  the  bound  of  the  optimal  objective  value,  given  in 
Eq.  (5.53),  of  Ok . 


Proof  of  Proposition  6.4. 

The  solution  to  the  master  problem  contains  a  nonnegative  combina¬ 
tion  of  at  most  n  +  1  columns  P1,  each  generated  by  a  control  func¬ 
tion  that  is  piecewise  constant,  and  each  has  at  most  (n+l)M  switch¬ 
ing  points  (Proposition  6.3).  Thus,  the  combination  of  the  controls  to 

2 

generate  the  solution  has  at  most  (n  +  1)  M  discontinuities.  The 


bound  (6  )  was  shown  in  Chapter  V. 


Q.E.D. 


Theorem  6,6.  The  optimal  control  generated  by  the  generalized  program¬ 
ming  solution  of  the  continuous-time  problem  for  the  linear  loss 
functionals  (minimum  fuel  and  minimal  time  problems  included  as 
special  cases)  is  a  piecewise  constant  function,  and  it  has  a 
finite  number  of  discontinuities. 


Proof  of  Theorem  6.6. 

If  the  generalized  program  terminates  with  a  value  of  6=0  for 

some  stage  k,  then  by  Proposition  6.4,  the  theorem  is  true. 

In  any  case,  given  the  optimal  dual  variables  to  the  generalized 
_* 

program  it  ,  the  optimal  solution  is  a  combination  of  at  most  n  +  1 
vectors  P1,  generated  as  solutions  to  (6.10).  The  generating  con¬ 
trols  of  these  vectors  have  at  most  M  switchings,  and  their  combina¬ 
tions  has  most  (n  +1)M  switchings. 

Q.  E.  D. 
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It  should  be  noted  that  when  5=0,  the  current  value  of  rr  is 
optimal,  and  it  can  be  used  to  determine  an  optimal  control  function  in¬ 
dependent  of  the  current  solution  (but  not  necessarily  distinct). 


Theorem  6.7.  If  the  solution  to  the  parametric  linear  programming  prob- 
lem  when  using  if  to  generate  /(t)  is  unique  except  on  a  set  of 
measure  zero,  the  optimal  control  function,  when  non-zero,  is  at  an 
extreme  point  of  the  admissible  control  region. 


Proof  of  Theorem  6.7. 

The  solution  to  a  linear  program  always  occurs  at  an  extreme  point 
of  the  constraint  set.  When  a  change  of  variables  is  made  to  produce  an 
equivalent  problem  for  minimum  fuel  problems,  a  control  of  level  zero  is 
considered  to  be  at  an  extreme  point  of  the  new  constraint  set.  Thus, 
the  optimal  control  is  at  an  extreme  point  of  the  admissible  control 
region  (or  an  equivalent  constraint  set  for  minimum  fuel  or  minimum  time 


problems) . 


Q.E.D. 


The  previous  theorem  also  implies  that  the  standard  minimal  time 
problem,  and  certain  linear  loss  problems,  have  bang-bang  solutions.  It 
also  implies  that  the  minimal  fuel  solution  is  a  bang-coast-bang  solu¬ 
tion  in  some  cases. 

Pontryagin  [1]  lias  shown  that  his  general  position  condition  is  a 
sufficient  condition  to  insure  that  the  parametric  linear  programming 
problem  has  a  unique  solution  almost  everywhere. 


Proposition  6.5.  The  upper  bound  on  the  number  of  switchings  for  the 
linear  loss  functionals  when  the  matrix  F  has  real  eigenvalues, 
the  state  of  the  system  is  n,  and  the  matrix  A  of  the  admissible 
control  region  is  m  X  p,  is 


(n  +  1)  (p  -  m) 
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Proof  of  Proposition  6.5. 

There  are  at  most  bases  for  the  parametric  linear  program. 

Each  has  (p  -  m)  nonbasic  variables  with  relative  cost  factors  y  (t) 

1 

having  at  most  (n  +  1)  points  at  which  it  becomes  value  zero.  Thus 
if  a  basis  can  be  repeated,  it  can  do  so  no  more  than  (n  +1)  (p  -  m) 
times,  after  which  it  remains  optimal.  Thus  each  column  P*  of  the 
master  program  is  generated  by  a  control  with  at  most 


(n  +  1) (p  -  m) 


switching  points.  Since  at  most  (n  +  1)  control  functions  are  com¬ 
bined,  the  maximum  number  of  switchings  is 


(n  +1)  (p  -  m) 


Q.E.D. 


C.  Characteristics  of  Quadratic  Loss  Optimal  Controls 

Since  the  parametric  quadratic  program  has  a  time  (parameter)  de¬ 
pendent  solution  for  the  control  vector,  the  only  characterization  of 
the  optimal  control  generated  is  in  the  class  of  time  functions  possible 
for  the  solution. 

For  the  quadratic  programming  problem,  stated  as 


w  =  Mz  +  q(t)  ,  w,  z  >  0,  wizi  =  0  i 


the  solution  has  the  form 


=  q(t)  +  M 


"  z 

w 


where 


=  q( t) ,  and  z,  w  =  0 
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WSm 


s 


q^(t)  is  merely  a  linear  combination  of  the  original  components  of  q(t) 
which  are  again  linear  combinations  of  the  components 

f  Skt 

t  e 


for  all  s  eigenvalues  of  F,  and  £  less  than  the  multiplicity 
of  the  eigenvalue  s  .  Thus  the  solution  to  the  parametric  quadratic 
program  is  composed  of  linear  combinations  of  the  same  elements. 

As  in  the  linear  case,  at  most  (n  +  1)  columns  of  the  generalized 
programming  master  problem  are  used  at  any  stage;  therefore,  the  current 
solution  is  of  the  same  form,  i.e.,  linear  combinations  of  the  elements 

„  s,  t 
k 

t  e 


Whenever  a  basis  switch  is  made,  the  linear  multipliers  change  in  the 
linear  combination,  but  the  solution  has  the  same  characteristic  form. 

Proposition  6.G.  At  any  stage  in  the  quadratic  control  problem,  includ¬ 
ing  the  optimal  solution,  the  form  of  each  component  of  the  control 
function  is  a  linear  combination  of  n  +  1  terms  of  the  form 

„  s  t 
,  £  k 
t  e  , 

with  only  the  constant  terms  changing  at  each  of  the  finite  number 
of  basis  switches  . 


Proof  of  Proposition  6.6. 

Each  column  generated  by  the  subproblem  has  a  control  function  of 
the  required  form  with  the  finite  number  of  basis  switches,  since  the 
control  function  is  generated  from  the  parametric  quadratic  program. 
Since  at  most  (n  +  1)  columns  are  combined  for  each  solution,  the 


current  control  solution  has  the  same  form.  At  the  optimum,  the  quad¬ 
ratic  program  has  a  unique  solution  for  positive  definite  Q,  and  only 
one  column  is  generated  with  the  generating  control,  which  is  optimal, 
having  the  required  form.  For  positive  semidefinite  Q,  the  solution 


M  " 


I 


to  the  quadratic  program  is  not  necessarily  unique,  and  a  combination 
of  at  most  (n  +  1)  control  functions  may  be  required  as  in  the  linear 
case. 

Q.  E.  D. 


This  algorithm  provides  an  open  loop  solution  to  the  optimal  control 
problem.  Also,  it  should  be  noted  that  no  assertion  is  made  regarding 
the  uniqueness  of  the  solution  in  the  form  of  the  optimal  control 
function. 

Since  Chapter  VII  provides  an  illustrative  example  with  computational 
experience  in  the  linear  case,  we  will  now  present  an  example  showing  the 
form  of  the  solution  for  the  case  of  quadratic  loss  in  control  [14], 
Consider  the  system 


x .  =  xo 
i  2 


x2  =  U  . 


Thus , 


*  0 

l  " 

’  0  ’ 

F  = 

G  = 

n 

o  _ 

_  1  _ 

xi<0)  =  0, 


x2(0)  =  0 


x^(T)  =  s1  ,  and  x2(T)  =  s2 


min 


u(  •) 


-1 


\  *  dt 


u  I  <1  , 


where  Q  =  —  I  and  is  positive  definite.  If  the  optimum  dual  variable 


to  the  generalized  programming  formulation  of  this  problem  is 


=  (-1.  rt*,  **,  , 
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the  optimal  solution  can  be  obtained  as  follows: 


find  the  control  u  (t)  so  that 


min  — 

u(t) |  <  1  * 


is  achieved  by  u  (t) 


Therefore,  u  (t)  must  solve 


12  *  *  F(T-t)  I  * 

2  u  +  «2)  e  Gu ( t )  j  +  rr3 


u ( t)  |  <  1,  t  <  [0,  T]  , 


min  )  1  2  * 

U(.)  I  2  u  +  ^l(1) ’  ^2(t)  Gu(t) 


u(t) |  <1  , 


where  tJi^Ct),  and  jr^Ct)  are  the  optimal  adjoint  variables  for  all  t. 


Hence  the  minimization  is 


12  * 
2  U  + 


■A 


u(t) |  <  1. 


For  this  problem,  the  solution  is  easily  seen  as 


u  (t)  =  -sat  \|r2 ( t)  = 


■)f  X- 

-t2(t)  .  |  to(t)  |  <_  1 


-sgn  \|f_(  t) ,  |  if  ( t)  |  >1 
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Note  that  ^(t)  has  the  form 


*  * 

(V  V 


=  n*(T-t)  +  it* 


1.  e.  , 


\|r2(t)  =  at  +  3  . 


Thus  the  optimal  solution  has  the  form  u(t)  =  at  +  3,  for  any  interval 

where  a  and  3  are  allowed  to  change  at  certain  switching  points. 

Free  Final  State  Problem.  In  conclusion  we  will  consider  the 

problem  where  the  initial  state  is  zero  and  the  final  state  (at  fixed 

th 

time,  T)  is  completely  free.  Since  the  first  through  n  rows  of  the 

generalized  programming  master  problem  have  free  right-hand  sides,  the 

slack  variables  for  these  rows  are  always  permitted  to  be  non-zero. 

— 

Thus,  the  optimal  it  must  be 


it  =  (1,  0,  . 


.  .  ,  it  , ) 
n+1 


The  optimal  control  is  then  determined  by  the  subproblem, 


f[x(u),  u] 


subject  to  u(t)  e  U  t  e  [0,T] 


Thus  the  problem  is  solved  by  one  iteration  of  a  parametric  programming 
problem. 
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Chapter  Vil 


EXAMPLES  AND  COMPUTATIONS 

In  this  chapter,  examples  are  used  to  show  the  execution 
results  of  the  algorithm.  The  convergence  properties  will  be 
strated  as  well  as  the  basic  features  of  the  algorithm. 

The  problem  we  intend  to  solve  is 


x.  =  x 
1  ^ 


X2  =  U 


u(t)|  < 


Therefore, 


'l  ' 

'  0  ' 

x(0)  = 

,  x(3)  = 

.0  . 

0 

— 

Thus 


‘  0 

1  ‘ 

‘  0  ' 

F  = 

,  G  = 

_  0 

0_ 

_  1  . 

Ft 


1 

0 


and 


F(T-t) 
e  G  = 


1 

0 


T-t 

1 


1 

1 - 

O 

1 _ 

I 

1 

c+ 

_ 1 

L  i  J " 

1 - 

M 

1 - 

-£ 


T-t 

1 


u(t)  dt  ; 


and  sample 
demon- 
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By  the  definitions  given  in  Chapter  V, 


sF  =  '  /T 

0  i  Lo 


1  3 


I L 0 


(L  oJi 


=  (s)  =  - 


We  initiate  by  looking  for  a  feasible  solution  to 


min  V1  +  \  - 

w  =  >  y  +  >  y 

p.y  L  1  Z,  1 


subject  to  Pm  +  Iy  -  Iy  = 


where 


(  r3  T_t  I 

p  e  c  =  {  P|  P  =  !  I  u(t)  tit, 

h  i 


u  ( t )  |  <  1  |  . 


0  0  ~ 

Lei  P  =  o  ’  generated  by  the  admissible  control  u(t)  =  0.  In 


this  example,  we  seek  to  solve  phase  I  by  keeping  the  desired  point  as 
the  right-hand  side  and  to  terminate  when  w  =  0. 
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The  first  master  problem  is  represented  by  the  tableau 


min 

n-y 


w 


2 

i=l 


+ 

y,  + 


2 


i=l 


P0  yl 

0  1 

0  0 

1  0 


+ 

^2 

0 

1 

0 


-1 

0 

0 


0 

-1 

0 


M 


O’ 


+ 


y.  >  0  . 

■’l  - 


-1 

0 

1 


The  first  two  rows  correspond  to  states  in  the  dynamic  system,  and  the 
final  row  represents  the  possibilities  of  convex  combinations  of  the 
columns  P1,  generated  by  the  subproblem.  The  optimal  basis  for  this 
linear  program  is 


B 


0 


'  0  0  -1  ' 
0  10 
10  0 


wi  th 

0  1  ' 
1  0 
0  0  _ 

The  solution  is 


with 


0 

w 


=  1  . 
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which  generates  a  vector  P  , 


i  r3 

- 1 

CO 

i 

rt 

_ 1 

1 

1 

CO 

cn 

_ 1 

p  = 

1 

u(t)  dt  = 

-1.0 

- 

—  _ 

The  new  master  problem  tableau  has  an  additional  column  correspond 
1 


ing  to  P  ,  i . e . , 


mi 

M 


<2  C 

in  v~  +  - 

w  =  \  y.  +  \  y. 

•  y  Z  1  Z  1 


M0  Mi 


0 

0 

1 


-3.  5 
-1.0 
1 


10-1 
0  10 

0  0  0 


0 

-1 

0 


p.,  yi.  y~±  >  o 


The  optimal  basis  for  this  linear  program  is 


B1  = 


-3,5  0  0 

-10  1 
110 


and  the  solution  is 


+ 

f2 


-1 

0 

1 


|.in  =  5/7,  p  =  2/7,  =  2/7,  all  others  equal  zero, 


w  =  2/7  . 
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Using  the  dual  variable  from  the  above  basis,  the  new  column  generated  by 
both  the  subproblem  and  the  generating  control  is 


P 


2 


4  .5 
3 


u2(t) 


1,  t  £  [0,3] 


The  new  tableau  is 


min 

p.y 


o 

o 

i 


^i 
-3  .5 
-1 
1 


2 

4  .5 
3 
1 


+ 

y. 


i 

o 

o 


2 

0 

1 

0 


yi 


-1 

0 

0 


J2 

0 

-1 

0 


Pi, 


y+,  yT  >  o 

i  i  — 


-l 

o 

i 


The  optimal  basis  for  this  linear  program  is 


B2  = 


0  -3.5  4.5 


-1 


1  J 


and  the  solution  is 


M-q  =  1/3,  41  =  1/2,  p2  = 


1/6 


wi  th 


2 


w 


=  0 


Thus,  a  feasible  control  has  been  found  for  the  control  problem. 
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•  *  •  rj  ■>  v  -  -  .*>  vr> v  v  v  \- hi 


or  in  tableau  form, 


max  A 

A  “o  V1  *2 

1  0  3  3  =0 

0  0  -3.5  4.5  =  -1 

0  0-13  =  0 

0  1  1  1  =  1 

U.  >  0  . 

l  - 

The  solution  to  this  linear  program  is 

=  1/3,  pi  =  1/2,  p2  =  1/6  , 

with 

Ak  =  2  . 

k 

If  we  define  u  as  the  control 

k 

— k  k  i  . 

u  =  >  Piu  (t)  , 

i=0 
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where  |_k  is  the  solution  to  the  k  master  program,  the  inequalities 
in  Theorem  5 ,6  become 


J(u*)  <  J(uk)  <  -Ak  =  Jk  . 


For  k  =  2  the  solution  is 


J(uk)  =  4/3  ,  -Ak  =  2 


-2 

The  dual  variable  jt  is 


7X2'  =  (1,  2,  -4,  0)  , 


and  the  subproblem  is 


. .  ,  tk  k+1  k+l 

find  o  ,  u  ,  P  ,  so  that 


5,k  -k ' 

6  =  mm  jt 

P  e  C 


P 

1 


or 


min 

u  ( t )  e  u 


f3  -  i 

Jo  |U 


(t)|  +  («k,  Ttk 


3-t 


u(t)  dt  +  JT, 


This  is  equivalent  to  finding  the  solution  to 


lu<t)|  +  [nk(3-t)  +  KkJ  u(t) 
j  u(t) |  <  1 

t  e  [0,3] 


For  k  =  2,  the  solution  is 
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v  r  w  r  vr  r  w  - 


-1 


t  £  [0,  1/2] 


u3(t) 


0  t  £  (1/2,  3/2] 

1  t  t  (3/2,  3] 


This  solution  produces  a  new  vector 


with 


53 


2  .0 
-0  .25 


1  .0 


1 


Thus,  by  using  the  inequalities  in  Theorem  5.6,  we  arrive  at 

J(uk)  +  0R  <  J(u*)  <  J(uR)  <  Jk  ; 
lor  k  =  2,  the  solution  is 

-1  .17  <  J(u*)  <  1  .33  . 

After  each  column  is  generated,  the  master  problem  is  augmented  (all  P1 
are  retained)  . 

We  will  now  present  the  results  from  a  computer  run  solving  the 
above  problem.  The  program  converged  in  40  iterations,  using  16  place  ac- 
curacy,  on  an  IBM  360/67.  Figure  1  illustrates  the  control  function  u  (t) 

at  iterations  corresponding  to  k  =  2,  3,  4,  5,  10,  15,  30,  40.  Its  cost 

_ lv  ,_k. 

J(u  )  is  shown  at  each  iteration  in  Fig.  2.  The  optimal  value  J(u  ) 

at  each  iteration  is  shown  in  Fig.  3,  and  the  distance  between  the  two 

k  ,  k 

curves  represents  the  magnitude  of  o  .  The  convergence  of  n  and  0 
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is  shown  in  Table  1 .  In  this  problem,  6  converged  monotonically  to 


zero.  The  convergence  of  n,  on  the  other  hand,  is  not  monotonic  by 
component  or  component-wise  norm.  However,  it  does  converge  on  a  sub¬ 
sequence  to  its  optimum  value  and  seems  to  monotonically  converge  in  the 
„  | _k 

norm  of  |  n  -  it  |  , 

A  common  penomenon  in  these  problems  was  observed  from  the  gener¬ 
ated  columns  and  their  corresponding  control  functions.  After  the  ini¬ 
tial  iterations,  the  new  columns  seemed  to  be  approaching  a  limit  and 
were  very  nearly  equal  component  wise.  This  is  due  to  the  uniqueness  of 
the  solution  to  the  subproblem  at  (or  near)  the  optimum  dual  variables. 
(Note  that  the  subproblem  has  a  unique  solution  for  every  stage  of  this 
problem.)  Thus  the  control  functions  are  converging  (as  seen  by  Fig.  1) 
to  their  optimum  value,  and  the  state  generated  by  these  controls  is 
converging  to  its  optimum  desired  value. 

This  similarity  in  the  generated  columns  produces  an  unusual  prob¬ 
lem  in  the  master  program.  The  master  program  develops  into  a  linear 
program  with  approximately  equal  columns  being  basic  or  "nearly"  basic 
columns.  Thus  the  basis  matrix  is  getting  closer  to  a  singular  matrix. 
For  computational  purposes,  this  activity  is  not  very  critical,  since 
it  only  occurs  when  optimality  is  close  at  hand,  and  termination  occurs 
before  the  basis  matrix  becomes  singular. 

lie  final  solution  computed  for  the  example  consists  of  a  control 


-1.000  t  fc  [0,  0.38196564] 

-0.1176  t  fc  (0.38196564,  0.38196754] 

-0.0784  t  fc  (0.38196754,  0.38196945] 

0.0  t  fc  (0.38196945,  2.61802864] 

0.0392  t  fc  (2.61802864,  2.61803246] 

0.1176  t  fc  (2.61803246,  2.61803436] 

1  .00  t  fc  (2 .61803436,  3  .0] 


* 

u  (t)  = 


with  a  cost  J(u  )  =  0.7639320.  If  accuracy  to  within  five  places  is 
sufficient,  then 
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Fig .  2  .  J(u  )  vs  k . 

Table  1 

CONVERGENCE  OF  DUAL  VARIABLES 


k 

TT 

1 

"2 

n3 

k 

o 

2 

2  .0 

-4.0 

0 

-2.5 

3 

1  .33 

-1 .67 

0 

-0  .83 

4 

1 .0 

-1 .75 

0 

-0  .31 

5 

0  .89 

-1  .22 

0 

-0  .14 

10 

0  .89 

-1.36 

0.12 

-0  .004 

15 

0  .89 

-1 .34 

0.128 

-0  .0001 

20 

0  .895 

-1  .34 

0 .131 

-0  .000004 

25 

0  .894 

-1.34 

0 .130 

-0  .00000013 

30 

0  .894 

-1  .34 

0  .130 

0 

35 

0  .8944 

-1 .3417 

0 .1305 

0 

40 

0  .8944 

-1 .3416 

0 .1305 

0 
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/ 

I 


J(uk)  +  8k 


0.0 


2  3  4  5 


20 


30 


-0.5 


t 

I 


1.0 


t 


-1.5 


Fig.  3. 


J(uk)  AND 


T,_k.  ck 
J(u  )  +  6  VS 


k  . 


* 

u  (t) 


!-l .0  t  e  [0,  0 .38197] 

0  t  e  (0.38197,  2.61803] 

1.0  t  e  (2 .61803,  3 .0] 


with  J  =  0  .7639 . 

To  compare  this  method  with  other  solution  procedures  for  the  mini¬ 
mum  fuel  problem,  we  observed  the  final  control  function  produces  a  fea¬ 
sible  control  which,  in  turn,  produces  an  objective  value  accurate  to 
15  digits  (double  precision  accuracy)  . 
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W: 


Although  linear  programming  can  be  used  in  the  discrete  version  of 
the  continuous  problem,  to  achieve  a  solution  as  good  as  that  obtained 
by  the  generalized  programming  method,  the  time  interval  would  have  to 
be  broken  into  more  than  one  million  increments;  these  increments  can 
produce  a  linear  program  with  over  a  million  variables  and  over  a  mil¬ 
lion  rows.  Naturally,  linear  programs  of  that  size  are  too  large  for 
existing  computers. 

By  using  the  generalized  programming  method,  only  linear  programs 
with  rows  approximately  equal  to  the  dimension  of  the  state  space  need 
to  be  5  olved  . 

Tne  concluding  example  illustrates  the  determination  of  the  exis¬ 
tence  of  a  feasible  solution  to  the  control  problem  to  solve  a  minimum 
time  problem. 


’  0 

1 

'  0  " 

Let 

F  = 

,  G  = 

.0 

0  . 

1  . 

"1  ' 

’  0  " 

x(0)  = 

,  x(T)  = 

_0  _ 

.0  „ 

mi 
u  ( 


r1 


dt 


with  I u ( t ) I  <  1  ,  Vt 


The  solution  to  this  problem  is  known  to  be  T  =  2.0  [15]  .  The  solu¬ 
tion  procedure  used  is  to  choose  some  very  large  T  and  solve  the  gen¬ 
eralized  program 


mm 

N.y 


w 


Ty,  -  Vy- 


Pp  +  iy.  -  iy.  =  S 

i  i 


=  1  , 


where 
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P  is  defined  as  previously.  If  the  problem  is  feasible,  reduce  T;  if 
not,  increase  T  and  continue. 


The  following  table  illustrates  the  number  of  iterations  required 

to  determine  a  feasible  solution,  if  it  exists,  or  the  infeasibility  of 

the  original  problem,  for  any  T.  Note  that  from  Theorem  5.2  when 
k  k 

w  +  b  >  0,  no  feasible  solution  exists  for  the  current  T. 


Time,  T 

of 

Number 

Iterations 

w 

b 

5 

2 

0 

0 

2  .05 

2 

0 

0 

2 

2 

0 

0 

1  .95 

2 

0  .05 

-0 .025 

1 

2 

0  .75 

-0  .25 

Thus,  for  times  when 

T  <  2 

.0, 

a  feasible 

solution 

can  be  found  after 

two  iterations;  and, 

for  times 

when  T  <  2 

.0,  the 

determination  of  an 

infeasible  problem  can  also 

be 

discovered 

after  two 

iterations . 
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An  algorithm  for  solving  Dantzig's  generalized  programming  formulation  of 
- o.itinuous-time  linear-system  optimal  control  problems  is  developed.  Dantzig's 
work  is  extended  to  include  continuous-time  versions  of  quadratic  loss  criteria 
and  minimum  fuel  problems.  New  results  in  parametric  linear  and  quadratic  program¬ 
ming  problems,  where  the  parameter  dependence  is  nonlinear,  are  derived  with 
internal  schemes  to  avoid  cycling  due  to  degeneracy.  Finite  switching  results  in 
the  completely  linear  system,  including  the  minimum  fuel  and  minimal  time  problems, 
are  presented  without  assuming  Pontryagin's  general  position  principal  or  unique¬ 
ness  properties.  The  procedure  initially  finds  a  feasible  and  admissible  solution 
to  the  continuous-time  problem  without  using  discrete  approximations.  The  algorithm 
continues  to  converge  monotonically  to  the  optimal  solution  while  remaining  feasible 
and,  at  each  stage,  provides  a  bound  on  the  value  of  the  loss  function  for  termina¬ 
tion  purposes.  This  procedure  is  well  suited  for  systems  with  a  relatively  high 
number  of  state  variables  and  control  inputs  for  which  discrete  time  linear  or 
quadratic  programming  models  become  too  large. 
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